I would like to have deeper comparison with alternatives like rtk, which are already fast and written in rust, also the previous comments mentioned something that has been a know problem with rtk that it sometimes strips the thing that the llm needs (or expects, causing more work to need to happan not less)
In term of token saving performance, it should be on par with rtk since it is basically the same idea.
The major different is rtk bundled hundreds of filter logic and no room for user to adjust without maintaing user owned fork or opening the pull request while lowfat is using opposite architectural approach by removing almost all filter logic in the binary and seperate user filters as a plugin system
I've tried rtx and lean-ctx and these tools seem to end up confusing the agent more than helping. Any saving is irrelevant if the agent decides to work around the tool and makes even more calls than it would otherwise.
I don't know about cost saving, but if it's keeping the context size down I've had a lot better results using subagents to keep a higher order conversation clean for longer.
The docs are missing any examples of what this does, instead showing _how_ it works - and only for the codebase itself, rather than the behavior of the app.
What would be useful:
- examples of text that can be filtered, and why that would be valuable
- a data flow diagram of runtime behavior, showing how filtering removes unnecessary context
Have terms been established to describe these types of tools? How do I refer to small utilities to perform specific transformations to LLM behavior? CLI filter seems pretty good to describe this tool conversationally but not so much when searching, they some low cardinality keywords.
I am thinking that a small tool that simply refuses to pass large CLI output to the LLM and warns it to filter the results before reading would achieve this better as the LLM would be forced into thinking and writting the filter itself.
It has the strip aggressiveness level suport.
You can tune up 3 levels for each template output of your stacktrace using lowfat-filter dsl, shellscript or python.
The LLMs already do that themselves with `tail` all the time. There's a lot of room for improvement on top of that. Though they usually figure it out after a few tries. I often just paste manual runs errors myself anyway.
It requires a bit effort in doing long-term adjustment and tuning for your agent common cli tools commands called. kinda need to evolve on day-to-day basis. But, agent itself can be useful to help tuning this.
I would like to have deeper comparison with alternatives like rtk, which are already fast and written in rust, also the previous comments mentioned something that has been a know problem with rtk that it sometimes strips the thing that the llm needs (or expects, causing more work to need to happan not less)
In term of token saving performance, it should be on par with rtk since it is basically the same idea. The major different is rtk bundled hundreds of filter logic and no room for user to adjust without maintaing user owned fork or opening the pull request while lowfat is using opposite architectural approach by removing almost all filter logic in the binary and seperate user filters as a plugin system
I've tried rtx and lean-ctx and these tools seem to end up confusing the agent more than helping. Any saving is irrelevant if the agent decides to work around the tool and makes even more calls than it would otherwise.
I don't know about cost saving, but if it's keeping the context size down I've had a lot better results using subagents to keep a higher order conversation clean for longer.
Subagents help with costs too, as they can run on much cheaper models.
The docs are missing any examples of what this does, instead showing _how_ it works - and only for the codebase itself, rather than the behavior of the app.
What would be useful:
Have terms been established to describe these types of tools? How do I refer to small utilities to perform specific transformations to LLM behavior? CLI filter seems pretty good to describe this tool conversationally but not so much when searching, they some low cardinality keywords.
I am thinking that a small tool that simply refuses to pass large CLI output to the LLM and warns it to filter the results before reading would achieve this better as the LLM would be forced into thinking and writting the filter itself.
Would this have any impact on the response quality from the agent?
How do you handle the risk of stripping out the exact stack trace the agent needed? That seems like the hard tradeoff here.
It has the strip aggressiveness level suport. You can tune up 3 levels for each template output of your stacktrace using lowfat-filter dsl, shellscript or python.
In a perfect world the LLM needs to be very explicit on what it wants to read
The LLMs already do that themselves with `tail` all the time. There's a lot of room for improvement on top of that. Though they usually figure it out after a few tries. I often just paste manual runs errors myself anyway.
gonna ask the same... do far it's has been manually choosing what's useful in each command for the agents?
It requires a bit effort in doing long-term adjustment and tuning for your agent common cli tools commands called. kinda need to evolve on day-to-day basis. But, agent itself can be useful to help tuning this.