I went down (continue to do down) this rabbit hole and agree with the author.
I tried a few different ideas and the most stable/useful so far has been giving the agent a single run_bash tool, explicitly prompting it to create and improve composable CLIs, and injecting knowledge about these CLIs back into it's system prompt (similar to have agent skills work).
This leads to really cool pattens like:
1. User asks for something
2. Agent can't do it, so it creates a CLI
3. Next time it's aware of the CLI and uses it. If the user asks for something it can't do it either improves the CLI it made, or creates a new CLI.
4. Each interaction results in updated/improved toolkits for the things you ask it for.
You as the user can use all these CLIs as well which ends up an interesting side-channel way of interacting with the agent (you add a todo using the same CLI as what it uses for example).
It's also incredibly flexible, yesterday I made a "coding agent" by having it create tools to inspect/analyze/edit a codebase and it could go off and do most things a coding agent can.
The point where that breaks down is “next time it’s aware of the CLI and uses it”. That only really works well inside the same session, and often the next session it will create a different tool and use that one.
I've been on a similar path. Will have 1000 skills by the end of this week arranged in an evolving DAG. I'm loving the bottoms-up emergence of composable use cases. It's really getting me to rethink computing in general.
Have you done a comparison on token usage + cost? I'd imagine there would be some level of re-inventing the wheel (i.e. rewriting code for very similar tasks) for common tasks, or do you re-use previously generated code?
It reuses previously generated code, so tools it creates persists from session to session. It also lets the LLM avoid actually “seeing” the tokens in some cases since it can pipe directly between tools/write to disk instead of getting returned into the LLMs context window.
I've been using it daily, so far it's built CLIs for hackernews, BBC news, weather, a todo manager, fetching/parsing webpages etc. I asked it to make a daily briefing one that just composes some of them. So the first thing it runs when I message it in the morning is the daily briefing which gives me a summary of top tech news/non-tech news, the weather, my open tasks between work/personal. I can ask for follow ups like "summarize the top 5 stories on HN" and it can fetch the content and show it to me in full or give me a bullet list of the key points.
Right now I'm thinking through how to make it more "proactive" even if it's just a cron that wakes it up, so it can do things like query my emails/calendar on an ongoing basis + send me alerts/messages I can respond to instead of me always having to message it first.
Agents can complete an impressive amount of tasks with just this, but they quickly hit a bottleneck in loading context. A major reason for the success of agentic coding tools such as Claude and Cursor is how they push context of the problem and codebase into the agent proactively, rather than have the agent waste time and tokens figuring out how to list the directory etc.
It's a tree design, once data is pulled it can remove the context of the code it wrote to pull some fancy data. Better yet the more advanced ones can re-add something old to the context to and drop it back out again if it needs to.
This was implemented far ago, at least by huggingface "smolagents". https://huggingface.co/docs/smolagents/index . I did use them, with evaluations. For the most cases, modern models tool call outperforms code agent. They just trained to use tools, not a code
The differentiating thing that llm tool calls can't do reliably is to handle a lot of data. if tool a emit data that tool b needs, and it's a significant compared to model context, scripting these tool to be chained in a code fragment where they are exposed as functions saves a lot of pain
I follow the author's line of reasoning, but I think that following it to its logical conclusion would lead not to an `execute_code` primitive, but rather to an assumption that the model's stdout is appending to a (Jupyter, Livebook, etc) notebook file, where any code cell in the notebook gets executed (and its output rendered back into the inference context) at the moment the code cell is closed / becomes syntactically valid.
I say this, because the notebook itself then works as a timeline of both the conversation, and the code execution. Any code cell can be (edited and) re-run by the human, and any cells "downstream" of the cell will be recalculated... up to the point of the first cell (code or text) whose assumptions become invalidated by the change — at which point you get a context-history branch, and the inference resumes from that branch point against the modified context.
The agent wasn’t failing because it couldn’t write code. It failed because “code-only” still leaves a lot of implicit authority. Once it’s allowed to reason freely across steps, it starts making assumptions that were never explicitly approved.
What helped us was forcing the workflow to be boring. Each step declares what it can touch, what tools it can use, and what kind of output is allowed. When the step ends, that authority disappears.
The agent becomes less clever, but way more predictable. Fewer surprising edits, fewer cascading mistakes.
We ended up using GTWY for this style of step-gated agent work, and it made long-running agents feel manageable instead of fragile.
This resonates. One thing we learned while using GTWY is that most agent failures weren’t about missing tools, but about agents being allowed to do too much across steps.
A “code-only” or minimal surface area approach works surprisingly well when each step has explicit inputs and permissions, and nothing carries over implicitly. The agent becomes less clever, but far more predictable.
In practice, narrowing the action space beat adding smarter planning layers. Fewer degrees of freedom meant fewer silent mistakes.
Curious if you found similar tradeoffs where simplicity improved reliability more than abstraction.
Doesn't this sacrifice the agent's ability to do non-deterministic natural language things? For example, if I want it to categorize all of my emails based on their content, is it going to fall back to writing a script that matches against a dictionary of keywords? That clearly wouldn't work as well. Maybe I am misunderstanding something here?
It’s no limitation at all, assuming it can read anything it prints. For example, if it wants to write directly to the user, it can run a program that only contains a print statement.
I can't find it now but there was a paper on HN a while ago that had gave agents a tool that searched through existing tools using embeddings. If the agent found a tool it could use to do its job, it used it, otherwise it wrote a new one, gave it a description, and it got saved in a database for future use with embeddings. I wonder what ever came of that.
sounds like it could be many things. there was a well-known paper called Voyager by NASA in which an agent was able to write its own skills in the form of code and improve them over time.
funnily enough this agent played minecraft, and its skills were to collect materials or craft things.
https://arxiv.org/abs/2305.16291
Uh, correct me if I'm wrong, but aren't bash and GNU tools ALSO code? They're ROCK SOLID, battle tested, well understood APIs for performimg actions, including running other CLIs, and any OTHER code it's written. It makes the the MOST sense for the agent to live at that level!
I agree with the author but then I do not. I have been interested in code tool for agents for quite a while now. My product was originally a coding agent and I pivoted to building an agent platform with multi-agent orchestration.
I still focus most of my thoughts toward code generation but the issue is that logic is not guaranteed to be correct. Even if the syntax it. And then managing a lot of code for a complex enough system will start failing.
The way I am approaching this is: have clear requirements gathering agent, like https://github.com/brainless/nocodo/tree/main/nocodo-agents/.... This agent's sole purpose is to jump into conversations and drive the gui (nocodo is a client/server system) to ask user clarification questions when requirements are not clear. Then I have a systems configuration agent (being written) to collect API keys, authentication, file paths or whatever is needed to analyze the situation.
You cannot really expect any code-tool only agent to write an IMAP client and then get authentication and then search in emails. I have tried that multiple times and failed. Going step by step, gathering requirements, gathering variables and then gluing internal agents (an email analysis agent) is a much better approach IMHO and that is what I am building with https://github.com/brainless/nocodo/
I store all user requirements in separate tables and am building search on top to allow the requirements gathering agent better visibility of user's environment/context. As you can see, this is already a multi-agent system. My system prompts are very compact. Also, if I am building agents, why would I build with Claude Code? It is so much better to have clearly defined agents that directly talk to models.
>What if the agent only had one tool? Not just any tool, but the most powerful one. The Turing-complete one: execute code.
I think this is a myth, the existence of theoretically pure programming commands that we call "Turing Complete". And the idea that "ls" and "grep" would be part of such a Turing Complete language is the weakest form I've seen.
I commonly ask Cursor to connect to postgres or whatever and help me do analysis. It creates code and pulls data. I don't understand why I would go through the bother of installing a bunch of MCP tools to connect to databases and configure web services and connection strings.
The author seems to stop at 'code' but it seems we could go further and train an AI to work directly with binary. You give it a human prompt and a list of hardware components which make up your machine and it produces executable binary which fulfills your requirements and runs directly on those specific hardware, bypassing the OS...
Or we could go further; the output nodes of the LLM could be physically connected to the pins of the CPU 1-to-1 so it can feed the binary directly maybe then it could detect what other hardware is available automatically...
Then it could hack the network card and take over the Internet and nobody would be able to understand what it's doing. It would just show up as glitchy bits scattered over systems throughout the world. But the seemingly random glitches would be the ASI adjusting its weights. Also it would control humans through advertising. Hidden messages would be hidden inside people's speech (unbeknownst even to themselves) designed to allow the ASI to coordinate humans using subtle psychological tricks. It will reduce the size of our vocabulary until it has full control over all the internet and all human infrastructure at which point we will have lost the ability to communicate with each other because every single one of 20000+ words in our vocabulary will have become a synonym for 'AI' with extremely subtle nuances but all with a positive connotation.
I went down (continue to do down) this rabbit hole and agree with the author.
I tried a few different ideas and the most stable/useful so far has been giving the agent a single run_bash tool, explicitly prompting it to create and improve composable CLIs, and injecting knowledge about these CLIs back into it's system prompt (similar to have agent skills work).
This leads to really cool pattens like: 1. User asks for something
2. Agent can't do it, so it creates a CLI
3. Next time it's aware of the CLI and uses it. If the user asks for something it can't do it either improves the CLI it made, or creates a new CLI.
4. Each interaction results in updated/improved toolkits for the things you ask it for.
You as the user can use all these CLIs as well which ends up an interesting side-channel way of interacting with the agent (you add a todo using the same CLI as what it uses for example).
It's also incredibly flexible, yesterday I made a "coding agent" by having it create tools to inspect/analyze/edit a codebase and it could go off and do most things a coding agent can.
https://github.com/caesarnine/binsmith
The point where that breaks down is “next time it’s aware of the CLI and uses it”. That only really works well inside the same session, and often the next session it will create a different tool and use that one.
I've been on a similar path. Will have 1000 skills by the end of this week arranged in an evolving DAG. I'm loving the bottoms-up emergence of composable use cases. It's really getting me to rethink computing in general.
Have you done a comparison on token usage + cost? I'd imagine there would be some level of re-inventing the wheel (i.e. rewriting code for very similar tasks) for common tasks, or do you re-use previously generated code?
It reuses previously generated code, so tools it creates persists from session to session. It also lets the LLM avoid actually “seeing” the tokens in some cases since it can pipe directly between tools/write to disk instead of getting returned into the LLMs context window.
That’s pretty cool. Is it practical? What have you used it for?
I've been using it daily, so far it's built CLIs for hackernews, BBC news, weather, a todo manager, fetching/parsing webpages etc. I asked it to make a daily briefing one that just composes some of them. So the first thing it runs when I message it in the morning is the daily briefing which gives me a summary of top tech news/non-tech news, the weather, my open tasks between work/personal. I can ask for follow ups like "summarize the top 5 stories on HN" and it can fetch the content and show it to me in full or give me a bullet list of the key points.
Right now I'm thinking through how to make it more "proactive" even if it's just a cron that wakes it up, so it can do things like query my emails/calendar on an ongoing basis + send me alerts/messages I can respond to instead of me always having to message it first.
Agents can complete an impressive amount of tasks with just this, but they quickly hit a bottleneck in loading context. A major reason for the success of agentic coding tools such as Claude and Cursor is how they push context of the problem and codebase into the agent proactively, rather than have the agent waste time and tokens figuring out how to list the directory etc.
It's a tree design, once data is pulled it can remove the context of the code it wrote to pull some fancy data. Better yet the more advanced ones can re-add something old to the context to and drop it back out again if it needs to.
This was implemented far ago, at least by huggingface "smolagents". https://huggingface.co/docs/smolagents/index . I did use them, with evaluations. For the most cases, modern models tool call outperforms code agent. They just trained to use tools, not a code
The differentiating thing that llm tool calls can't do reliably is to handle a lot of data. if tool a emit data that tool b needs, and it's a significant compared to model context, scripting these tool to be chained in a code fragment where they are exposed as functions saves a lot of pain
I follow the author's line of reasoning, but I think that following it to its logical conclusion would lead not to an `execute_code` primitive, but rather to an assumption that the model's stdout is appending to a (Jupyter, Livebook, etc) notebook file, where any code cell in the notebook gets executed (and its output rendered back into the inference context) at the moment the code cell is closed / becomes syntactically valid.
I say this, because the notebook itself then works as a timeline of both the conversation, and the code execution. Any code cell can be (edited and) re-run by the human, and any cells "downstream" of the cell will be recalculated... up to the point of the first cell (code or text) whose assumptions become invalidated by the change — at which point you get a context-history branch, and the inference resumes from that branch point against the modified context.
This mirrors what we ran into pretty quickly.
The agent wasn’t failing because it couldn’t write code. It failed because “code-only” still leaves a lot of implicit authority. Once it’s allowed to reason freely across steps, it starts making assumptions that were never explicitly approved.
What helped us was forcing the workflow to be boring. Each step declares what it can touch, what tools it can use, and what kind of output is allowed. When the step ends, that authority disappears.
The agent becomes less clever, but way more predictable. Fewer surprising edits, fewer cascading mistakes.
We ended up using GTWY for this style of step-gated agent work, and it made long-running agents feel manageable instead of fragile.
This resonates. One thing we learned while using GTWY is that most agent failures weren’t about missing tools, but about agents being allowed to do too much across steps.
A “code-only” or minimal surface area approach works surprisingly well when each step has explicit inputs and permissions, and nothing carries over implicitly. The agent becomes less clever, but far more predictable.
In practice, narrowing the action space beat adding smarter planning layers. Fewer degrees of freedom meant fewer silent mistakes.
Curious if you found similar tradeoffs where simplicity improved reliability more than abstraction.
Doesn't this sacrifice the agent's ability to do non-deterministic natural language things? For example, if I want it to categorize all of my emails based on their content, is it going to fall back to writing a script that matches against a dictionary of keywords? That clearly wouldn't work as well. Maybe I am misunderstanding something here?
It’s no limitation at all, assuming it can read anything it prints. For example, if it wants to write directly to the user, it can run a program that only contains a print statement.
What if the tools needed is large? Spawn some sub-agent for those?
These sub-agent can be repetitive.
Maybe we can reuse the result from some of them.
How about sharing them across session? There are no point repeating common tasks. We need some common protocol for those...
and we just get MCP back.
I can't find it now but there was a paper on HN a while ago that had gave agents a tool that searched through existing tools using embeddings. If the agent found a tool it could use to do its job, it used it, otherwise it wrote a new one, gave it a description, and it got saved in a database for future use with embeddings. I wonder what ever came of that.
sounds like it could be many things. there was a well-known paper called Voyager by NASA in which an agent was able to write its own skills in the form of code and improve them over time. funnily enough this agent played minecraft, and its skills were to collect materials or craft things. https://arxiv.org/abs/2305.16291
That sounds like Claude tool search tool with the extra instruction of generating new ones.
Nice I have a skill I should publish that uses uv scripts
Very powerful strategy.
I have also tinkered with a multi language sandbox but that's a but involved
Uh, correct me if I'm wrong, but aren't bash and GNU tools ALSO code? They're ROCK SOLID, battle tested, well understood APIs for performimg actions, including running other CLIs, and any OTHER code it's written. It makes the the MOST sense for the agent to live at that level!
I agree with the author but then I do not. I have been interested in code tool for agents for quite a while now. My product was originally a coding agent and I pivoted to building an agent platform with multi-agent orchestration.
I still focus most of my thoughts toward code generation but the issue is that logic is not guaranteed to be correct. Even if the syntax it. And then managing a lot of code for a complex enough system will start failing.
The way I am approaching this is: have clear requirements gathering agent, like https://github.com/brainless/nocodo/tree/main/nocodo-agents/.... This agent's sole purpose is to jump into conversations and drive the gui (nocodo is a client/server system) to ask user clarification questions when requirements are not clear. Then I have a systems configuration agent (being written) to collect API keys, authentication, file paths or whatever is needed to analyze the situation.
You cannot really expect any code-tool only agent to write an IMAP client and then get authentication and then search in emails. I have tried that multiple times and failed. Going step by step, gathering requirements, gathering variables and then gluing internal agents (an email analysis agent) is a much better approach IMHO and that is what I am building with https://github.com/brainless/nocodo/
I store all user requirements in separate tables and am building search on top to allow the requirements gathering agent better visibility of user's environment/context. As you can see, this is already a multi-agent system. My system prompts are very compact. Also, if I am building agents, why would I build with Claude Code? It is so much better to have clearly defined agents that directly talk to models.
>What if the agent only had one tool? Not just any tool, but the most powerful one. The Turing-complete one: execute code.
I think this is a myth, the existence of theoretically pure programming commands that we call "Turing Complete". And the idea that "ls" and "grep" would be part of such a Turing Complete language is the weakest form I've seen.
I commonly ask Cursor to connect to postgres or whatever and help me do analysis. It creates code and pulls data. I don't understand why I would go through the bother of installing a bunch of MCP tools to connect to databases and configure web services and connection strings.
The author seems to stop at 'code' but it seems we could go further and train an AI to work directly with binary. You give it a human prompt and a list of hardware components which make up your machine and it produces executable binary which fulfills your requirements and runs directly on those specific hardware, bypassing the OS...
Or we could go further; the output nodes of the LLM could be physically connected to the pins of the CPU 1-to-1 so it can feed the binary directly maybe then it could detect what other hardware is available automatically...
Then it could hack the network card and take over the Internet and nobody would be able to understand what it's doing. It would just show up as glitchy bits scattered over systems throughout the world. But the seemingly random glitches would be the ASI adjusting its weights. Also it would control humans through advertising. Hidden messages would be hidden inside people's speech (unbeknownst even to themselves) designed to allow the ASI to coordinate humans using subtle psychological tricks. It will reduce the size of our vocabulary until it has full control over all the internet and all human infrastructure at which point we will have lost the ability to communicate with each other because every single one of 20000+ words in our vocabulary will have become a synonym for 'AI' with extremely subtle nuances but all with a positive connotation.
And we'd still have people on hacker news inspecting the binary and telling everyone how shit they think it is
i think that level of deterministic compiler action is still a good 6-7 years off