If you use a deterministic sampling strategy for the next token (e.g., always output the token with the highest probability) then a traditional LLM should be deterministic on the same hardware/software stack.
This is amazing, yet frightening because I'm sure someone will actually attempt to use it. It's like vibe coding on steroids.
- Each time you import a module, the LLM generates fresh code
- You get more varied and often funnier results due to LLM hallucinations
- The same import might produce different implementations across runs
There are a few thresholds of usefulness for this. Right now it’s a gimmick. I can see a world in a few years or maybe decades in which we almost never look at the code just like today we almost never look at compiled bytecode or assembly.
There's not much of a world in which we don't check up and verify what humans are doing to some degree periodically. Non-deterministic behavior will never be trusted by default, as it's simply not trustable. As machines become more non-deterministic, we're going to start feeling about them in similar ways we already feel about other such processes.
you'd be surprised, but there's actually a bunch of problems you can solve with something like this, as long as you have a safe place to run the generated code
> Not suitable for production-critical code without review
Ah, dang it! I was about to deploy this to my clients... /s
Otherwise, interesting concept. Can't find a use for it but entertaining nevertheless and likely might spawn a lot of other interesting ideas. Good job!
Wow, what a nightmare of a non-deterministic bug introducing library.
Super fun idea though, I love the concept. But I’m getting the chills imagining the havoc this could cause
It's like automatically copy-pasting code from StackOverflow, taken to the next level.
Sounds like a fun way to learn effective debugging.
Are there any stable output large language models? Like stablediffusion does for image diffusion models.
If you use a deterministic sampling strategy for the next token (e.g., always output the token with the highest probability) then a traditional LLM should be deterministic on the same hardware/software stack.
Deterministic is one thing, but stable to small perturbations in the input is another.
It imports the bugs as well. No human involvement needed. Automagically.
This is amazing, yet frightening because I'm sure someone will actually attempt to use it. It's like vibe coding on steroids.
There are a few thresholds of usefulness for this. Right now it’s a gimmick. I can see a world in a few years or maybe decades in which we almost never look at the code just like today we almost never look at compiled bytecode or assembly.
There's not much of a world in which we don't check up and verify what humans are doing to some degree periodically. Non-deterministic behavior will never be trusted by default, as it's simply not trustable. As machines become more non-deterministic, we're going to start feeling about them in similar ways we already feel about other such processes.
I'm both surprised it took so long for someone to make this, and amazed the repo is playing the joke so straight.
Possibly the funniest part is the first example being a totp library
you'd be surprised, but there's actually a bunch of problems you can solve with something like this, as long as you have a safe place to run the generated code
> from autogenlib.antigravity
As a joke, that doesn't feel quite so far-fetched these days. (https://xkcd.com/353/)
can it run Doom tho?
Of course, this code was generated by ChatGPT.
This is horrifying
I love it
nooooo the side project ive put off for 3 years
From now on you'll be able to just do `import side_project` until it works.
Can it input powerpoint slides?
looks very fun excited to try it out
Hysterical, I like that caching is default off because it's funnier that way heh
LOL
Need your explanation.
> Not suitable for production-critical code without review
Ah, dang it! I was about to deploy this to my clients... /s
Otherwise, interesting concept. Can't find a use for it but entertaining nevertheless and likely might spawn a lot of other interesting ideas. Good job!