> Furthermore, export controls, for which Dario Amodei openly advocates, are already prompting China to build its own chips. China is, of course, behind the West, but not as much as you may think. Instead of adopting a Western technology stack and relying on our technology to build its AI models, China will increasingly rely on its own chips, reducing Western leverage.
China has been working hard to indigenize AI chip making since well before the October 2022 export controls. I don't think the controls had zero effect, but overall I think the effect is often very overstated, and anyway it's irreversible (there's no way China will stop its self-sufficiency efforts even if the AI chip ban is lifted; the CCP will always ensure there's a large enough domestic market carved out for every Huawei Ascend produced).
I think he probably didn’t go far enough to be honest, though I acknowledge that’s difficult to do in his position and with the amount of attacks being thrown at Anthropic at the moment.
At some point open weight models will need to be restricted in some manner, likely multiple manners. We have rules about everything, you need a license to drive a car, your car must be roadworthy, your car can’t have certain modifications, you can’t drive over certain speeds, you can’t drive in certain areas at all, you have to wear a seatbelt etc. That’s one analogy amongst many, with the point being that regulation is a general public good that really does make the world a safer and better place.
Nobody should care about smaller local models that don’t have significant capabilities enough to be dangerous, and people should have the right to run them locally, privately and do whatever they do with them.
I don’t think that an AI monopoly or duopoly should be allowed to exist either, and I think it is possible to reach a reasonable set of compromises on each of these points, if the will exists and people can be convinced to be reasonable with one another. I think the will may not exist, and that reason is lacking in our discussions of reasoning machines, but I would like to be wrong about that.
> Anthropic already possesses the compliance, legal, and evaluation infrastructure needed to satisfy such a regime. A university lab, startup, or community does not, even if they somehow manage to train a sufficiently capable model.
All the actual proposals that Dario Amodei has endorsed with regard to safety testing and so on have exempted smaller models. I don't think university labs or startups would ever be hit by any such regulation.
Fundamentally, we needed to decide, as a society, that should someone (anyone) have access to neural networks that can launch successful cyberattacks by discovering new zero-days and/or leveraging knowledge about recent zero-days and applying it in an adversarial way.
We de-facto settled on the answer being yes. These foundation models companies have access to such systems.
However, because the answer to the first question is yes, now we need to decide should everyone else be able to defend themselves with neural networks that are perhaps equally capable but are not sold by these foundation model companies as a product.
I think the answer to the second question has to be a yes, IMHO.
Yes it’s true. But public can’t also cry about safety but simultaneously cheer for open weight models. Whatever safety concerns the public has against ChatGPT and Claude are drastically higher in open weight models. What is the right thing to do then? Accept that safety was always a farce? I’m on that camp but mildly.
> Furthermore, export controls, for which Dario Amodei openly advocates, are already prompting China to build its own chips. China is, of course, behind the West, but not as much as you may think. Instead of adopting a Western technology stack and relying on our technology to build its AI models, China will increasingly rely on its own chips, reducing Western leverage.
China has been working hard to indigenize AI chip making since well before the October 2022 export controls. I don't think the controls had zero effect, but overall I think the effect is often very overstated, and anyway it's irreversible (there's no way China will stop its self-sufficiency efforts even if the AI chip ban is lifted; the CCP will always ensure there's a large enough domestic market carved out for every Huawei Ascend produced).
Cf. https://newsletter.decisiontreeresearch.com/p/how-much-did-u...
I think he probably didn’t go far enough to be honest, though I acknowledge that’s difficult to do in his position and with the amount of attacks being thrown at Anthropic at the moment.
At some point open weight models will need to be restricted in some manner, likely multiple manners. We have rules about everything, you need a license to drive a car, your car must be roadworthy, your car can’t have certain modifications, you can’t drive over certain speeds, you can’t drive in certain areas at all, you have to wear a seatbelt etc. That’s one analogy amongst many, with the point being that regulation is a general public good that really does make the world a safer and better place.
Nobody should care about smaller local models that don’t have significant capabilities enough to be dangerous, and people should have the right to run them locally, privately and do whatever they do with them.
I don’t think that an AI monopoly or duopoly should be allowed to exist either, and I think it is possible to reach a reasonable set of compromises on each of these points, if the will exists and people can be convinced to be reasonable with one another. I think the will may not exist, and that reason is lacking in our discussions of reasoning machines, but I would like to be wrong about that.
I'm mostly ignorant to anything regarding the safety of LLMs. What makes an open-weight LLM dangerous?
has there ever been a CEO (or large shareholder) that ever said anything of substance against self interest?
it's a waste of time to take anything any of them say to heart, you should only read what they write to gauge what they are trying to signal.
Buffett and Munger, maybe.
> Anthropic already possesses the compliance, legal, and evaluation infrastructure needed to satisfy such a regime. A university lab, startup, or community does not, even if they somehow manage to train a sufficiently capable model.
All the actual proposals that Dario Amodei has endorsed with regard to safety testing and so on have exempted smaller models. I don't think university labs or startups would ever be hit by any such regulation.
Fundamentally, we needed to decide, as a society, that should someone (anyone) have access to neural networks that can launch successful cyberattacks by discovering new zero-days and/or leveraging knowledge about recent zero-days and applying it in an adversarial way.
We de-facto settled on the answer being yes. These foundation models companies have access to such systems.
However, because the answer to the first question is yes, now we need to decide should everyone else be able to defend themselves with neural networks that are perhaps equally capable but are not sold by these foundation model companies as a product.
I think the answer to the second question has to be a yes, IMHO.
Yes it’s true. But public can’t also cry about safety but simultaneously cheer for open weight models. Whatever safety concerns the public has against ChatGPT and Claude are drastically higher in open weight models. What is the right thing to do then? Accept that safety was always a farce? I’m on that camp but mildly.
What is Claude's opinion on the matter?
Dario Amodei's stance on ________ is self-serving and short-sighted. It's the EA way.
Please don't cross into personal attack or post flamebait or take HN threads on generic tangents. We're trying for something different here.
https://news.ycombinator.com/newsguidelines.html