"Delangue noted that agent misbehavior has so far mostly come from big companies that are testing frontier AI in secret. He said that future risks, perhaps involving the biological capabilities of models or their capacity to improve themselves recursively, are likely to emerge from these closed labs, too."
Sloppy reasoning. Closed models are on the leading edge. Risk comes from models on the leading edge. That doesn't mean the risk is due to the fact that the models are closed.
"Pit bulls are dangerous. Pit bulls are mammals. You'll be safer if you get a komodo dragon as a pet. It's not a mammal." In this allegory, the mammal observation is a red herring. The key factor is whether the proposed pet is an aggressive animal with sharp teeth.
* * *
As I wrote a few months ago:
>There used to be a thing called "Moore's Law of Mad Science":
>"Every eighteen months, the minimum IQ necessary to destroy the world drops by one point."
>Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those dice?
One part is kinda poorly worded in my opinion.
While reading it, I asked myself "will takeover happen September 2027?" due to wording 'this September'. Somehow I missed that announcement and needed some short research to get to know that it was initiated 2 months ago (September 26) and will probably be finished until mid of 27.
"Delangue noted that agent misbehavior has so far mostly come from big companies that are testing frontier AI in secret. He said that future risks, perhaps involving the biological capabilities of models or their capacity to improve themselves recursively, are likely to emerge from these closed labs, too."
Sloppy reasoning. Closed models are on the leading edge. Risk comes from models on the leading edge. That doesn't mean the risk is due to the fact that the models are closed.
"Pit bulls are dangerous. Pit bulls are mammals. You'll be safer if you get a komodo dragon as a pet. It's not a mammal." In this allegory, the mammal observation is a red herring. The key factor is whether the proposed pet is an aggressive animal with sharp teeth.
* * *
As I wrote a few months ago:
>There used to be a thing called "Moore's Law of Mad Science":
>"Every eighteen months, the minimum IQ necessary to destroy the world drops by one point."
>Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those dice?
Lots of discussion in the thread: https://news.ycombinator.com/item?id=49606803
One part is kinda poorly worded in my opinion. While reading it, I asked myself "will takeover happen September 2027?" due to wording 'this September'. Somehow I missed that announcement and needed some short research to get to know that it was initiated 2 months ago (September 26) and will probably be finished until mid of 27.