Here's a guy who does more than make simple, bloated end-user apps. He tries to dig into the code and make novel performance optimizations etc. He sees that AI can't code everything.
I liked the two original experiments in the article, which in a microcosm gels with my 1+ year of deep agentic development. But I would like a whole article about his experience, and with which model (Opus? Fable? erg, Sonnet?) and effort he used.
I've progressed in using latest Claude-kins and the GPTs as usually competent teammate/buddies, and generally know to sort out the fluff confidence with the realz (shoot, that was how I was when I was but a wee little coder lad: overconfident because of an error-free compile and one non-segfault run.)
You have to put in the time, the skill creation, the system prompt/personalization, the (sometimes adversarial) automation, the testing, verification, kicking down the loop castles (as usually caused by being cheeky with highest effort levels.)
I have been doing this (vibecoding) for as long as the term vibecoding was coined. The amount of people I have met that describe the process like you did and then proceed to show the most soulless vibecoded app is staggering.
> In a past life I had to argue every year to renew a license for a profiling tool that cost about 20 EUR a month. I’ve heard since that everyone at the company is now getting a Claude subscription, even non-programmers.
Underrated quote. I also found this frustrating. At one company I was at (a very old company which was trying to pivot to software engineering), we had hellish bureaucratic fights with the IT department to get access to Pycharm, Obsidian, and even GitHub. But then the AI craze dropped and management just gave us all GitHub Copilot access without us even asking.
Cory Doctorow's book The Reverse Centaur's Guide to Life After AI talks more about this. It's just a modern symptom of an age-old power struggle. The workers want more control over their craft, including quality standards and tools, but their bosses want more control over the workers.
What's happening now is bosses are feeling pressure from investors to show productivity gains from using AI, so bosses panic-push AI within their companies. Which leads to misaligned incentives like tokenmaxxing.
The author is a real C++ programmer. Not like the PyTorch guys who use std::shrared_ptr for cyclical graphs and then go on to overpromise on threaded Python only to find out that the slowdown is much larger and everything is brittle as expected years ago.
The times that Meta people can browbeat honest engineers are over.
Programming by hand also empircally leads to quite a lot of bugs. The actual calculus for whether these tools are worth it to solve a given problem is as function of how much time it takes to reach the desired level of quality compared to writing by hand. I struggle to see why "they hallucinate and aren't deterministic" is enough to conclude that it doesn't ever end up being worth it when there's no alternative that doesn't also produce some bugs.
I don't particularly like these tools, but I also can't deny that for some things I'm genuinely able to get to the results I want faster with them than without. Yes, some people will use them to produce low quality software, but in my experience it's mostly people who already would have been producing low quality software as well, just at a much slower pace. Having a higher volume of low quality software is a problem, but it's distinct from the claim that the tools not ever being useful in producing high quality software.
It's possible that from reading my rebuttals you'll think I'm one of the people using these tools to produce low quality software, but it's not clear how to falsify that claim. If that's the conclusion you'd draw, then you probably weren't really open to considering rebuttals in the first place though.
If the author can’t put in the effort to think of something original to say that hasn’t been said 1000x, why should I put the effort into rebutting cliches that were last relevant in 2024?
> Hallucinations are an inherent property of how LLMs work.
While technically true the hallucination rates on modern models is low and other checks can ensure that by the time a human sees it it is most likely solid.
For research there is more danger as there is less feedback loop other than other LLM scrutinising the first. For research I get it to come to a conclusion but provide me with links so I can judge. More like advanced search.
“While technically true the hallucination rates on modern models is low…”
Isn’t this entirely context dependent? Where did you get the information that modern models have low hallucination rates? I’d love to see the benchmark if there is one, it seems like it would be useful to track.
Hence the advice to read the tests? Interesting how people make the argument that AI code is garbage and can't be trusted (it's inelegant and poorly structured, sure, but that's a different matter), and how the only option isn't to just spot check the work, but to laboriously write it by hand.
Reading tests and actually catching issues requires a heck of a lot of effort. IME code reviews of tests are often more laborious than reviewing the code itself since so much if it is reasoning about corner cases.
I'm not saying don't do it, but the LOE is high. Done well I'd argue it runs close to the effort involved in just authoring those tests by hand.
And that's ignoring that the "vibe all the things" crowd is explicitly telling people not to read the code. At all.
(And if you think I'm exaggerating: an exec where I work decided to own rebuilding one of our services. They told me how recently the LLM generated multiple thousands of lines of code and they pushed it with minimal review, figuring we'll just fix the bugs as they happen...).
Rebuttals to what? Ill-faith rhetorics and circular judgement?
Points 1-2 were clearly cited not because of their factual content, but because they boil down to the author mocking the existence of industry trends, and writing them off whole (incl. this one). This is unsound, because it violates the principle of suspending (prior) judgement: https://en.wikipedia.org/wiki/Suspension_of_judgment - it sets both the author and their readers up for a specific conclusion, rather than inspiring nonbias. Hopefully one does not need to explain why this is problematic?
It is further incredibly trite to bring up how AI is an ongoing trend (self-evident), and the fact that trends distort perception (also self-evident). Them fighting fire with fire and bringing their own gut judgement is not any more intellectually respectable.
Point 3 is completely unsupported and misleading. The author probably means that hallucinations are formally unavoidable, but that on its own doesn't carry much weight, and is not the same thing.
Point 4 is factually wrong. AI is an entire academic field with a half a century of history to its name, and this is trivial to learn. Clearly not why it was cited once again either however, but because it's blatantly ill faith too.
I'm not disappointed in the article, there are many of them out there. I'm disappointed that this is near the top of HN. I lazily outsource my information hunter and gathering to HN.
It'd be nice if blog authors categorized their posts into "old man shaking fist at clouds" vs "something I searched for on the internet, legitimately never found it, and decided to write it myself"
Article was flagged from the front page after criticism of Meta. You know, if free speech isn't allowed maybe we should contact Zuckerberg directly if that is what you want.
At the risk of sharing an unpopular take, this reads a lot like someone who decided how they feel about LLM-driven engineering ~5 months ago and doesn’t seem to reflect the current reality.
(Or they’re working in an organization with lower budgets and not cranking the frontier models of today)
I fully agree about the cost/sustainability parts, but to suggest you can’t build a high quality coding/verifying/iterating loop for _most_ problems is disingenuous.
His experience is completely plausible. He’s in a niche that requires highly performant code and most complex, highly performant games do nit have source available for models to train on. It’s a very common observation that the farther you stray from mainstream, the less effective the LLM models become.
Have you actually tried performance optimisation using an agent?
With any programming language/framework that has quality profiling tooling (which is a prerequisite for most projects) I have had huge success with automated hotspot profiling where the LLM can propose theories, test the impact of fixes, convince you of which to pursue, etc.
High performance algorithms are quite well documented so it isn't unreasonable to expect an LLM to apply them appropriately when given the ability to "see" where they need to be applied.
LLMs work perfectly fine for me but I’m building web app equivalents of binder keepers, like most people. Essentially store data, display data. The novelty is in what/how we’re displaying. LLMs are owning this market.
Unless you yourself are doing commercial game development, calling the author disingenuous puts your own pro-AI bias on display. It’s based on speculation. I prefer to take his very specific examples and experiments at face value.
I’ve specifically used Opus to diagnose and fix performance bottlenecks in parallel Rust code on multiple occasions (e.g improving NPS for a chess engine) and it works well.
I’ve done plenty of performance architecting in my day-job and rule #1 is generally “you can’t fix what you can’t see/measure”. I have a suspicion that many folks aren’t investing in letting AI actually introspect iterative execution via the appropriate harness, and are then acting surprised that it is no oracle.
To be honest, the optimization the author talks about isn't really high level optimization. And Claude's pattern actually calls for a more extensible design. Strictly speaking, the author's instruction could be considered incorrect. I'm not trying to dismiss the author.
If performance were truly critical, they wouldn't have been using Unity in the first place. They would have used Unreal, as mentioned earlier. And if they were sticking with Unity, they would have tried ECS.
Unity is fundamentally based on the template method pattern. The idea of pulling Update out and handling it in a single manager class is really more of a small scale indie game approach. It's a technique that scales very poorly.
In practice, there are many better optimization techniques for GameUpdateable. So I'm not sure why this particular example was used to demonstrate performance optimization.
Typically, you could use GameUpdateable with object pooling, which would be a safer approach. There are also many batching techniques available.
In other words, this isn't about performance. It's a technique used for small indie game development. By handling it directly through a manager, registration and removal no longer depend on the Unity framework and become manually scheduled by the user. This, in turn, means you have to handle many more edge cases, which creates additional work. This is a common pattern, sacrificing future extensibility for immediate performance gains.
It's a technique used in small indie games. Converting per frame Update callbacks into a central loop that iterates over all objects is where GameUpdateable would actually be a better choice. So rather than viewing this as an optimization for performance, it should be understood as a design choice made to make small games easier to manage.[1]
> this reads a lot like someone who decided how they feel about LLM-driven engineering ~5 months ago and doesn’t seem to reflect the current reality.
I've been reading this for 2 years straight. "oh you have a criticism of AI? Well they fixed that in Aeternos v Y-point-Z, which after doing all of my work also gave my wife an orgasm for the first time this year, obviously OP is using the old model".
5 months ago people said the exact same things, you know. That this stuff was how it was 5 months before that, and doesn't reflect the current reality.
Moreso it seems like nonsense that's masked by an excellent facade of eloquence. Ironic given it's a a post ranting against LLMs.
I stopped reading when he decided to go off on some tangent about how LLMs can't do native programming or games because they weren't trained on it. Ignoring the technical issues there, an even more overt one is that, amongst a zillion other projects, even the entire source code for Unreal Engine is available and within their training corpus. Even older models were quite competent at working with Unreal and outputting idiomatic code, which is saying something if anybody's ever worked with Unreal.
I increasingly think people are writing dumb articles on purpose because it drives 'engagement' more than a straight forward and accurate post would, at least on average.
> I do admit that this approach immediately triggered my contrarian side and made me very defiant of any AI tool.
Makes sense.
> While this could be partially remedied by always asking for a primary source or citation, I dislike the idea that one has to add magical incantations to their queries to get the right results. It’s a good laugh to make fun of “make no mistake” memes, until you start having to consider similar things seriously.
Man I am genuinely stumped at the obvious lack of desire to use something in a way it's supposed to be used. LLMs are tools and like with any tool it's on us to use it properly, not hitting a screw with a hammer and saying that hammers are a very stupid tool.
It just seems silly to need to beg in English for things like that. What's the entire space of things you need to think about? Can you be sure the specific way you asked would be correct, or do you also have to add "don't lie about following this instruction:"? Why not have an interface `ai --make-no-mistakes --dont-make-things-up`? I don't want to have to talk to the computer like it's people.
But English is literally the interface. It's the *same* as `--make-no-mistakes`, it's just your preference compared to directly telling it to the AI which is ok. That doesn't change the fact the using the tool correctly != using the tool.
You don't have to. Chatgpt-sol-high already does that for me. Could be the extra instructions or the base system prompt or whatever, point is - it already does it.
> You shouldn't be required to tell an information retrieval tool to actually retrieve information rather than making it up!
And therein lies the problem: perception. LLMs are treated like information retrieval tools but in reality are probability machines that return plausible/mostly accurate information.
They're not searching, they're inferring and then guessing. That the guesses are often quite good means that we can easily fool ourselves with whatever it spits out. We call it hallucination but it's a feature, not a bug.
We're not talking about "the public". The person writing the article is a very experienced software engineer. There shouldn't be a gap in understanding.
To be fair, hammers are bad screwdrivers. They are faster at driving screws and require less finesse to do so, but are really bad at taking them out, often break the screw, and often result in a fastening that superficially looks correct but fails to perform well under stress.
Two new types of screw, the ring-shank nonhelical fastener and the rivet, however, are changing everything. Soon, using screwdrivers will be rare except in special circumstances.
I don't really see your point. You're still using the screwdriver and it's up to you to use it when it makes sense, see if it's doing it's job properly, fix issues it creates etc... If it's not doing a good job for your particular screw don't use it. But don't say it's a horrible tool since you choose to not use it properly.
There is a middle ground between CEOisms "LLMs are literally Jesus" and people similar to this blog's author "LLMs are mostly trash."
Part of the entire premise of the article is that many people aren’t being given a choice, so this is a really weird deflection. I disagree about the author not recognizing a middle ground.
Not providing a source is how Llms get caught in sycophancy and hallucinations. The “tool” needs improvement, especially one that claims it’s intelligence. How would you react if you found out after schooling everything you learned has half-hallucinated because your teacher made it up and there is no text book or historical log of facts? You had assumed this whole time you were learning facts…now what?
Again the analogy isn't complete. If I went in knowing the teacher isn't an expert in the field and they might get some things wrong I'd definitely double check stuff. I'm sure you've had similar teachers in your life and you did this either way (I know I have).
I read through a bit of it, and it seems like the author assumes training was done only on open source data, but I doubt that's actually the case. There are plenty of paid, proprietary source codes being sold out there, hacked code, that is.
Aside from that, I don't agree with the author's view on agentic workflows. Modern AI native development runs like a massive state machine, starting from MCP, local file systems, and what's usually called a harness.
I also noticed what might be a mistake in the author's domain, games. Putting aside the fact that inheritance based OOP is an outdated pattern, the suggestion to remove update() and put it into a manager class's List, then iterate with a for loop, is meant to eliminate overhead like P/Invoke costs in C#. But if Foo is still a class, a reference type, then List<Foo> is just an array of pointers scattered across heap memory. Pointer chasing can still happen. So I think that's actually bad advice.(Of course, the same issue exists in Mr. Claude's code as well.)
If the author truly wanted Data Oriented Programming(or DOD), they would have specified struct arrays or NativeArray.
This is a tricky area. The author's goal was likely to remove the per MonoBehaviour Update call cost in Unity, which is why they suggested using List<Foo>. But the instruction seems ambiguous, and I think that's part of the difficulty with AI development.
The objective function is the same, but the implementation varies and subtly differs from what I actually want.
From a design perspective, for team maintenance, the GameUpdateable abstraction might actually be better. But it's difficult. In terms of extensibility, an update manager that handles registration and expansion of multiple update targets might be over abstracting.
Writing this down makes me realize how many things I actually consider when putting code into a program. Sometimes I model how my next teammate might read it, and sometimes my words might be interpreted differently. It's really difficult.
The case study of the LLM getting the basic Mono framework usage wrong seems outdated.
They're pretty flawless at any framework now, React, SwiftUI, Unity, what-have-you.
If an LLM can write a nuanced paragraph about any topic, it can certainly write a simple [insert popular framework] component which has considerably less potential variation.
Consider that there are 100k+ words in English, 90k+ in Spanish, and 77 in C# (38 in JavaScript!).
The LLM can write code.
Can it make software? No. Because software is a lot more than code.
But the LLM can write code.
Any anti-AI takes moving forward are going to have to acknowledge that I think
Here's a guy who does more than make simple, bloated end-user apps. He tries to dig into the code and make novel performance optimizations etc. He sees that AI can't code everything.
[dead]
I liked the two original experiments in the article, which in a microcosm gels with my 1+ year of deep agentic development. But I would like a whole article about his experience, and with which model (Opus? Fable? erg, Sonnet?) and effort he used.
I've progressed in using latest Claude-kins and the GPTs as usually competent teammate/buddies, and generally know to sort out the fluff confidence with the realz (shoot, that was how I was when I was but a wee little coder lad: overconfident because of an error-free compile and one non-segfault run.)
You have to put in the time, the skill creation, the system prompt/personalization, the (sometimes adversarial) automation, the testing, verification, kicking down the loop castles (as usually caused by being cheeky with highest effort levels.)
I have been doing this (vibecoding) for as long as the term vibecoding was coined. The amount of people I have met that describe the process like you did and then proceed to show the most soulless vibecoded app is staggering.
> In a past life I had to argue every year to renew a license for a profiling tool that cost about 20 EUR a month. I’ve heard since that everyone at the company is now getting a Claude subscription, even non-programmers.
Underrated quote. I also found this frustrating. At one company I was at (a very old company which was trying to pivot to software engineering), we had hellish bureaucratic fights with the IT department to get access to Pycharm, Obsidian, and even GitHub. But then the AI craze dropped and management just gave us all GitHub Copilot access without us even asking.
Cory Doctorow's book The Reverse Centaur's Guide to Life After AI talks more about this. It's just a modern symptom of an age-old power struggle. The workers want more control over their craft, including quality standards and tools, but their bosses want more control over the workers.
What's happening now is bosses are feeling pressure from investors to show productivity gains from using AI, so bosses panic-push AI within their companies. Which leads to misaligned incentives like tokenmaxxing.
It reads like something written by a time traveler from the past.
We are all time travelers from the past.
> “Agentic workflow” (or whatever they’re calling it at the time you’re reading this article)
> Unlike the silver bullets of the past (like microservices or NoSQL)
> Hallucinations are an inherent property of how LLMs work.
> It’s all marketing and buzzwords
Not a serious article or thinker. I can get this stuff on Reddit if I want to read thrice-regurgitated cliches about AI.
The author is a real C++ programmer. Not like the PyTorch guys who use std::shrared_ptr for cyclical graphs and then go on to overpromise on threaded Python only to find out that the slowdown is much larger and everything is brittle as expected years ago.
The times that Meta people can browbeat honest engineers are over.
Nice corporate take.
I'd like to hear actual rebuttals to those quotes.
Programming by hand also empircally leads to quite a lot of bugs. The actual calculus for whether these tools are worth it to solve a given problem is as function of how much time it takes to reach the desired level of quality compared to writing by hand. I struggle to see why "they hallucinate and aren't deterministic" is enough to conclude that it doesn't ever end up being worth it when there's no alternative that doesn't also produce some bugs.
I don't particularly like these tools, but I also can't deny that for some things I'm genuinely able to get to the results I want faster with them than without. Yes, some people will use them to produce low quality software, but in my experience it's mostly people who already would have been producing low quality software as well, just at a much slower pace. Having a higher volume of low quality software is a problem, but it's distinct from the claim that the tools not ever being useful in producing high quality software.
It's possible that from reading my rebuttals you'll think I'm one of the people using these tools to produce low quality software, but it's not clear how to falsify that claim. If that's the conclusion you'd draw, then you probably weren't really open to considering rebuttals in the first place though.
If the author can’t put in the effort to think of something original to say that hasn’t been said 1000x, why should I put the effort into rebutting cliches that were last relevant in 2024?
So you basically agree but complain there are no new arguments against AI programming in the article?
No, I’m not sure where you are getting the idea that I agree with what I am referring to as cliches last relevant in 2024.
How can it be both irrelevant and cliché?
what's the point of making a ragebait comment if you didn't like the article? Whatever you think about the post, your reaction is much lower quality.
> Hallucinations are an inherent property of how LLMs work.
While technically true the hallucination rates on modern models is low and other checks can ensure that by the time a human sees it it is most likely solid.
For research there is more danger as there is less feedback loop other than other LLM scrutinising the first. For research I get it to come to a conclusion but provide me with links so I can judge. More like advanced search.
“While technically true the hallucination rates on modern models is low…”
Isn’t this entirely context dependent? Where did you get the information that modern models have low hallucination rates? I’d love to see the benchmark if there is one, it seems like it would be useful to track.
> it it is most likely solid.
Most likely? That’s not reassuring at all. So you’re saying the other checks can result in hallucinations?
If you're using an LLM and not an agentic workflow then you're doing things wrong in 2026.
My agents have to pass tests meaning that if their LLM hallucinates, the agent tools capture it and not me.
Some people have the agent hallucinating its own meaningless tests
Hence the advice to read the tests? Interesting how people make the argument that AI code is garbage and can't be trusted (it's inelegant and poorly structured, sure, but that's a different matter), and how the only option isn't to just spot check the work, but to laboriously write it by hand.
> Hence the advice to read the tests?
Reading tests and actually catching issues requires a heck of a lot of effort. IME code reviews of tests are often more laborious than reviewing the code itself since so much if it is reasoning about corner cases.
I'm not saying don't do it, but the LOE is high. Done well I'd argue it runs close to the effort involved in just authoring those tests by hand.
And that's ignoring that the "vibe all the things" crowd is explicitly telling people not to read the code. At all.
(And if you think I'm exaggerating: an exec where I work decided to own rebuilding one of our services. They told me how recently the LLM generated multiple thousands of lines of code and they pushed it with minimal review, figuring we'll just fix the bugs as they happen...).
Rebuttals to what? Ill-faith rhetorics and circular judgement?
Points 1-2 were clearly cited not because of their factual content, but because they boil down to the author mocking the existence of industry trends, and writing them off whole (incl. this one). This is unsound, because it violates the principle of suspending (prior) judgement: https://en.wikipedia.org/wiki/Suspension_of_judgment - it sets both the author and their readers up for a specific conclusion, rather than inspiring nonbias. Hopefully one does not need to explain why this is problematic?
It is further incredibly trite to bring up how AI is an ongoing trend (self-evident), and the fact that trends distort perception (also self-evident). Them fighting fire with fire and bringing their own gut judgement is not any more intellectually respectable.
Point 3 is completely unsupported and misleading. The author probably means that hallucinations are formally unavoidable, but that on its own doesn't carry much weight, and is not the same thing.
Point 4 is factually wrong. AI is an entire academic field with a half a century of history to its name, and this is trivial to learn. Clearly not why it was cited once again either however, but because it's blatantly ill faith too.
See also: https://en.wikipedia.org/wiki/Motivated_reasoning
3,300 words but nothing new to say.
Dislikes AI so much and ironically wrote a whole article without doing any of their own thinking.
I'm not disappointed in the article, there are many of them out there. I'm disappointed that this is near the top of HN. I lazily outsource my information hunter and gathering to HN.
Maybe you guys just shouldn't read blogs.
It'd be nice if blog authors categorized their posts into "old man shaking fist at clouds" vs "something I searched for on the internet, legitimately never found it, and decided to write it myself"
The blogger owes you nothing.
Article was flagged from the front page after criticism of Meta. You know, if free speech isn't allowed maybe we should contact Zuckerberg directly if that is what you want.
At the risk of sharing an unpopular take, this reads a lot like someone who decided how they feel about LLM-driven engineering ~5 months ago and doesn’t seem to reflect the current reality.
(Or they’re working in an organization with lower budgets and not cranking the frontier models of today)
I fully agree about the cost/sustainability parts, but to suggest you can’t build a high quality coding/verifying/iterating loop for _most_ problems is disingenuous.
His experience is completely plausible. He’s in a niche that requires highly performant code and most complex, highly performant games do nit have source available for models to train on. It’s a very common observation that the farther you stray from mainstream, the less effective the LLM models become.
Have you actually tried performance optimisation using an agent? With any programming language/framework that has quality profiling tooling (which is a prerequisite for most projects) I have had huge success with automated hotspot profiling where the LLM can propose theories, test the impact of fixes, convince you of which to pursue, etc.
High performance algorithms are quite well documented so it isn't unreasonable to expect an LLM to apply them appropriately when given the ability to "see" where they need to be applied.
LLMs work perfectly fine for me but I’m building web app equivalents of binder keepers, like most people. Essentially store data, display data. The novelty is in what/how we’re displaying. LLMs are owning this market.
Unless you yourself are doing commercial game development, calling the author disingenuous puts your own pro-AI bias on display. It’s based on speculation. I prefer to take his very specific examples and experiments at face value.
I’ve specifically used Opus to diagnose and fix performance bottlenecks in parallel Rust code on multiple occasions (e.g improving NPS for a chess engine) and it works well.
I’ve done plenty of performance architecting in my day-job and rule #1 is generally “you can’t fix what you can’t see/measure”. I have a suspicion that many folks aren’t investing in letting AI actually introspect iterative execution via the appropriate harness, and are then acting surprised that it is no oracle.
To be honest, the optimization the author talks about isn't really high level optimization. And Claude's pattern actually calls for a more extensible design. Strictly speaking, the author's instruction could be considered incorrect. I'm not trying to dismiss the author.
If performance were truly critical, they wouldn't have been using Unity in the first place. They would have used Unreal, as mentioned earlier. And if they were sticking with Unity, they would have tried ECS.
Unity is fundamentally based on the template method pattern. The idea of pulling Update out and handling it in a single manager class is really more of a small scale indie game approach. It's a technique that scales very poorly.
In practice, there are many better optimization techniques for GameUpdateable. So I'm not sure why this particular example was used to demonstrate performance optimization.
Typically, you could use GameUpdateable with object pooling, which would be a safer approach. There are also many batching techniques available.
In other words, this isn't about performance. It's a technique used for small indie game development. By handling it directly through a manager, registration and removal no longer depend on the Unity framework and become manually scheduled by the user. This, in turn, means you have to handle many more edge cases, which creates additional work. This is a common pattern, sacrificing future extensibility for immediate performance gains.
It's a technique used in small indie games. Converting per frame Update callbacks into a central loop that iterates over all objects is where GameUpdateable would actually be a better choice. So rather than viewing this as an optimization for performance, it should be understood as a design choice made to make small games easier to manage.[1]
[1]https://docs.unity3d.com/Manual/events-per-frame-optimizatio...
> this reads a lot like someone who decided how they feel about LLM-driven engineering ~5 months ago and doesn’t seem to reflect the current reality.
I've been reading this for 2 years straight. "oh you have a criticism of AI? Well they fixed that in Aeternos v Y-point-Z, which after doing all of my work also gave my wife an orgasm for the first time this year, obviously OP is using the old model".
5 months ago people said the exact same things, you know. That this stuff was how it was 5 months before that, and doesn't reflect the current reality.
Moreso it seems like nonsense that's masked by an excellent facade of eloquence. Ironic given it's a a post ranting against LLMs.
I stopped reading when he decided to go off on some tangent about how LLMs can't do native programming or games because they weren't trained on it. Ignoring the technical issues there, an even more overt one is that, amongst a zillion other projects, even the entire source code for Unreal Engine is available and within their training corpus. Even older models were quite competent at working with Unreal and outputting idiomatic code, which is saying something if anybody's ever worked with Unreal.
I increasingly think people are writing dumb articles on purpose because it drives 'engagement' more than a straight forward and accurate post would, at least on average.
> An Honest Review
> I do admit that this approach immediately triggered my contrarian side and made me very defiant of any AI tool.
Makes sense.
> While this could be partially remedied by always asking for a primary source or citation, I dislike the idea that one has to add magical incantations to their queries to get the right results. It’s a good laugh to make fun of “make no mistake” memes, until you start having to consider similar things seriously.
Man I am genuinely stumped at the obvious lack of desire to use something in a way it's supposed to be used. LLMs are tools and like with any tool it's on us to use it properly, not hitting a screw with a hammer and saying that hammers are a very stupid tool.
It just seems silly to need to beg in English for things like that. What's the entire space of things you need to think about? Can you be sure the specific way you asked would be correct, or do you also have to add "don't lie about following this instruction:"? Why not have an interface `ai --make-no-mistakes --dont-make-things-up`? I don't want to have to talk to the computer like it's people.
But English is literally the interface. It's the *same* as `--make-no-mistakes`, it's just your preference compared to directly telling it to the AI which is ok. That doesn't change the fact the using the tool correctly != using the tool.
You don't have to. Chatgpt-sol-high already does that for me. Could be the extra instructions or the base system prompt or whatever, point is - it already does it.
How are they supposed to be used? The providers promised us we could just say what we wanted and get it.
It's more like a hammer that never drives nails straight unless you hold it in a specific uncomfortable manner.
You shouldn't be required to tell an information retrieval tool to actually retrieve information rather than making it up!
> You shouldn't be required to tell an information retrieval tool to actually retrieve information rather than making it up!
And therein lies the problem: perception. LLMs are treated like information retrieval tools but in reality are probability machines that return plausible/mostly accurate information.
They're not searching, they're inferring and then guessing. That the guesses are often quite good means that we can easily fool ourselves with whatever it spits out. We call it hallucination but it's a feature, not a bug.
> It's more like a hammer that never drives nails straight unless you hold it in a specific uncomfortable manner.
I mean sure, pick any analogy, still doesn't change the fact that there are right uses and wrong uses; right ways to use it and wrong ways to use it.
> information retrieval tool
How is an LLM this? It's a non-deterministic word generator.
An LLM is not what the public was “sold”, they were “sold” a personified agent. That is the gap.
We're not talking about "the public". The person writing the article is a very experienced software engineer. There shouldn't be a gap in understanding.
To be fair, hammers are bad screwdrivers. They are faster at driving screws and require less finesse to do so, but are really bad at taking them out, often break the screw, and often result in a fastening that superficially looks correct but fails to perform well under stress.
Two new types of screw, the ring-shank nonhelical fastener and the rivet, however, are changing everything. Soon, using screwdrivers will be rare except in special circumstances.
I don't really see your point. You're still using the screwdriver and it's up to you to use it when it makes sense, see if it's doing it's job properly, fix issues it creates etc... If it's not doing a good job for your particular screw don't use it. But don't say it's a horrible tool since you choose to not use it properly.
There is a middle ground between CEOisms "LLMs are literally Jesus" and people similar to this blog's author "LLMs are mostly trash."
Part of the entire premise of the article is that many people aren’t being given a choice, so this is a really weird deflection. I disagree about the author not recognizing a middle ground.
> Part of the entire premise of the article is that many people aren’t being given a choice
Yeah, "part", but that's not the part I wrote about?
Not providing a source is how Llms get caught in sycophancy and hallucinations. The “tool” needs improvement, especially one that claims it’s intelligence. How would you react if you found out after schooling everything you learned has half-hallucinated because your teacher made it up and there is no text book or historical log of facts? You had assumed this whole time you were learning facts…now what?
Again the analogy isn't complete. If I went in knowing the teacher isn't an expert in the field and they might get some things wrong I'd definitely double check stuff. I'm sure you've had similar teachers in your life and you did this either way (I know I have).
I read through a bit of it, and it seems like the author assumes training was done only on open source data, but I doubt that's actually the case. There are plenty of paid, proprietary source codes being sold out there, hacked code, that is.
Aside from that, I don't agree with the author's view on agentic workflows. Modern AI native development runs like a massive state machine, starting from MCP, local file systems, and what's usually called a harness.
I also noticed what might be a mistake in the author's domain, games. Putting aside the fact that inheritance based OOP is an outdated pattern, the suggestion to remove update() and put it into a manager class's List, then iterate with a for loop, is meant to eliminate overhead like P/Invoke costs in C#. But if Foo is still a class, a reference type, then List<Foo> is just an array of pointers scattered across heap memory. Pointer chasing can still happen. So I think that's actually bad advice.(Of course, the same issue exists in Mr. Claude's code as well.)
If the author truly wanted Data Oriented Programming(or DOD), they would have specified struct arrays or NativeArray.
This is a tricky area. The author's goal was likely to remove the per MonoBehaviour Update call cost in Unity, which is why they suggested using List<Foo>. But the instruction seems ambiguous, and I think that's part of the difficulty with AI development.
The objective function is the same, but the implementation varies and subtly differs from what I actually want.
From a design perspective, for team maintenance, the GameUpdateable abstraction might actually be better. But it's difficult. In terms of extensibility, an update manager that handles registration and expansion of multiple update targets might be over abstracting.
Writing this down makes me realize how many things I actually consider when putting code into a program. Sometimes I model how my next teammate might read it, and sometimes my words might be interpreted differently. It's really difficult.
The case study of the LLM getting the basic Mono framework usage wrong seems outdated.
They're pretty flawless at any framework now, React, SwiftUI, Unity, what-have-you.
If an LLM can write a nuanced paragraph about any topic, it can certainly write a simple [insert popular framework] component which has considerably less potential variation.
Consider that there are 100k+ words in English, 90k+ in Spanish, and 77 in C# (38 in JavaScript!).
The LLM can write code.
Can it make software? No. Because software is a lot more than code.
But the LLM can write code.
Any anti-AI takes moving forward are going to have to acknowledge that I think