Author here. I'm running offline LLMs and using openrouter. I observed that many harnesses stumble over their communication with them: network errors, malformed tool calls, hallucinated response tags, empty responses etc. which leads to aborted flows and user-visible errors. I wrote llm-gauze as a transparent openAI compatible proxy that works around many of these issues.
Author here. I'm running offline LLMs and using openrouter. I observed that many harnesses stumble over their communication with them: network errors, malformed tool calls, hallucinated response tags, empty responses etc. which leads to aborted flows and user-visible errors. I wrote llm-gauze as a transparent openAI compatible proxy that works around many of these issues.