So like- I considered this idea of using a fast decision model to prune before compaction, but I scrapped it because A; im not bottlenecked by speed, and B; cache invalidation dark magic
From my understanding, if we have turns [A, B, C, D, E] and we decide that turn C is useless context, then we lose our cache for the [D, E] segment. (plz correct me if im wrong, I havent tested this)
Personally, I pay no mind to session state nowadays. I exclusively use provider native hosted compaction, which if you believe the providers, is the way to go. For context management- I spend most of my effort on the things which dump output back to the model.
If anyone has any good reads on benchmarking OpenAI and Anthropic hosted/native compaction against custom stuff, I would love to read!
> The tea task for FireRed was in its weights, so it poisoned its context.
Mmm, to my mind, isn't this mixing completely different things? I thought poisoned context is stuff in the context that makes the model behave poorly. But if it's in the weights, that's just... model issues.
>This is the advice I keep giving now. Someone says, “My coding agent’s doing something weird. Claude’s been nerfed.” And I ask, “How big is your context?”
That’s really true, but on the other hand, if there’s too little information, it also ends up working in strange ways. Most AGENT code anti-patterns work well at a medium amount of context, but when there’s too much or too little information, they don’t behave properly.
So like- I considered this idea of using a fast decision model to prune before compaction, but I scrapped it because A; im not bottlenecked by speed, and B; cache invalidation dark magic
From my understanding, if we have turns [A, B, C, D, E] and we decide that turn C is useless context, then we lose our cache for the [D, E] segment. (plz correct me if im wrong, I havent tested this)
Personally, I pay no mind to session state nowadays. I exclusively use provider native hosted compaction, which if you believe the providers, is the way to go. For context management- I spend most of my effort on the things which dump output back to the model.
If anyone has any good reads on benchmarking OpenAI and Anthropic hosted/native compaction against custom stuff, I would love to read!
> The tea task for FireRed was in its weights, so it poisoned its context.
Mmm, to my mind, isn't this mixing completely different things? I thought poisoned context is stuff in the context that makes the model behave poorly. But if it's in the weights, that's just... model issues.
>This is the advice I keep giving now. Someone says, “My coding agent’s doing something weird. Claude’s been nerfed.” And I ask, “How big is your context?”
That’s really true, but on the other hand, if there’s too little information, it also ends up working in strange ways. Most AGENT code anti-patterns work well at a medium amount of context, but when there’s too much or too little information, they don’t behave properly.