I love the concept, but tried reading the README on the got, but I am still unclear about precisely what it does or how it does it. Some more detail would be welcome.
I think that "infer once, repeat resulting logic forever" is one of the most important / sustainable / humane ways to imagine how AI acclimates to the universe.
I was working with a large entries on Batch inference, and yes LLMs are quite expensive to process Billions of contracts at latency, so I in my orignial paper was on the expensive thinking once, run the cheap deterministic result forever.
Hi HN. STROT compiles a decision task — "route these tickets", "score this feedback", "classify this text" — into a standalone pipeline. A large LLM is used once, at compile time, to synthesize the labels, the input projection, and the handlers. At runtime a small local model (144M, CPU) makes every decision: ~60ms/record, $0 marginal, zero LLM calls, bounded typed-JSON output.
The goal was to move the expensive, nondeterministic part (an LLM) entirely to build time and keep runtime frozen, cheap, local, and auditable — the way a compiler moves work off the hot path.
What I think is actually interesting, beyond "small model does inference":
- The compiler measures instead of guessing. It scores candidate labelings on a teacher-labeled sample and keeps the winner; on the identical model that's +18pp on a routing benchmark (92.5% vs 74.8%) and +14pp on text classification (86.7% vs 72.7%) over naive integration.
- It fits a conformal-calibration threshold at compile time, so low-confidence records abstain with a marginal error guarantee rather than guessing.
- A golden-record gate rejects any compile that fails known-answer examples — synthesis is stochastic, and I show that variance in the benchmarks.
What it doesn't do: it can't make a small model understand things it fundamentally can't. On tweet hate-speech it hits the model's ceiling; the fix there is a task-trained model in the swappable slot, not the compiler. All of it — including the losses — is in the repo's benchmarks with reproducible scripts.
There's a browser demo (a 144M model ( which is changeable ) playing Pac-Man, one decision per frame) and a quickstart that runs in a few lines. MIT licensed, pure-stdlib core.
Happy to go deep on the calibration math, the compile/cache design, or where this does and doesn't make sense.
I love the concept, but tried reading the README on the got, but I am still unclear about precisely what it does or how it does it. Some more detail would be welcome.
I think that "infer once, repeat resulting logic forever" is one of the most important / sustainable / humane ways to imagine how AI acclimates to the universe.
https://pickipedia.xyz/wiki/PickiPedia:Artificial_Dumb
I was working with a large entries on Batch inference, and yes LLMs are quite expensive to process Billions of contracts at latency, so I in my orignial paper was on the expensive thinking once, run the cheap deterministic result forever.
> If a ghost is close (nearest_ghost_distance small, " "within 2 steps), choose flee_ghost. Otherwise choose " "chase_dot to eat the nearest dot."
Dude, this is AI psychosis. Just ask the LLM to write you a program (“compile”) and then you can run it locally forever
That is literally what this does
Hi HN. STROT compiles a decision task — "route these tickets", "score this feedback", "classify this text" — into a standalone pipeline. A large LLM is used once, at compile time, to synthesize the labels, the input projection, and the handlers. At runtime a small local model (144M, CPU) makes every decision: ~60ms/record, $0 marginal, zero LLM calls, bounded typed-JSON output.
The goal was to move the expensive, nondeterministic part (an LLM) entirely to build time and keep runtime frozen, cheap, local, and auditable — the way a compiler moves work off the hot path.
What I think is actually interesting, beyond "small model does inference":
- The compiler measures instead of guessing. It scores candidate labelings on a teacher-labeled sample and keeps the winner; on the identical model that's +18pp on a routing benchmark (92.5% vs 74.8%) and +14pp on text classification (86.7% vs 72.7%) over naive integration. - It fits a conformal-calibration threshold at compile time, so low-confidence records abstain with a marginal error guarantee rather than guessing. - A golden-record gate rejects any compile that fails known-answer examples — synthesis is stochastic, and I show that variance in the benchmarks.
What it doesn't do: it can't make a small model understand things it fundamentally can't. On tweet hate-speech it hits the model's ceiling; the fix there is a task-trained model in the swappable slot, not the compiler. All of it — including the losses — is in the repo's benchmarks with reproducible scripts.
There's a browser demo (a 144M model ( which is changeable ) playing Pac-Man, one decision per frame) and a quickstart that runs in a few lines. MIT licensed, pure-stdlib core.
Happy to go deep on the calibration math, the compile/cache design, or where this does and doesn't make sense.
> Happy to go deep on the calibration math, the compile/cache design, or where this does and doesn't make sense.
yes plz
>On tweet hate-speech it hits the model's ceiling
This sounds like a sus endeavor to redline your LLM with, could you elaborate ?
Was this entirely vibe coded? The readme is very obviously Claude with no human edits at all.
Yes, even the comment you replied to is very noticeably claude authored.