This seems like as a good an opportunity as any to break out the Computer Fraud and Abuse Act.
They want “regulation” but we already have it. Hacking is illegal. Start locking up those responsible for this mess and I assure you they’ll “have a handle on it” quite quickly.
I still can't fathom my company application security decision.
Found pretty damning requests in our logs. Escalated. Expected it would result in at least reporting the TOS break from the originating place (one of cloud providers). Instead of that, they just went: "yeah, but we don't have logs". Provided them. "Yeah, but that ip doesn't resolve". I matched real ones from the load balancer. "There could just be many of them". There was one. When I had all the evidence gathered, they looked at me and finally told:
cfaa heavily relies on intent for prosecution (hence why researchers arent typically locked up). it would be difficult to argue that openai intended to hack other companies.
there's probably better/more likely to succeed avenues to pursue rather than the cfaa
They could make that argument the first time it happened but the next time or the fifth time I don't see how they can still credibaly claim to not know that the computer they control was going to do that
That would require mens rea. This may be criminal negligence, but it wouldn't be more than that. (I am basically 100% sure none of the employees or execs are intending or desiring any of these outcomes, regardless of the very large number of people who believe in conspiracy theories about regulatory capture and other sinister motives.)
I'm totally on board with treating it as gross negligence requiring hundreds of millions or billions of dollars paid in fines and compensation to victims, but don't act like this is more than what it is.
can we legally treat ai companies like parents of children? if a child drives a car into a storefront, the parent is responsible for the damage, and at some point you might even criminally charge the parent if there was gross negligence
The law already works like that. You can’t write some computer code and automation, set it loose, and then go “oops the computer did it.” It’s not a defense. While it may be hard to nail down which specific engineers to lock up, the corporation as a whole absolutely can be charged criminally.
As Mitt Romney once said “corporations are people.”
IMO it's because they don't want to handle their rogue AI activity. They want regulation around AI where they will inevitably be the beneficiaries even if they are the initial target. They can then lobby regulation in their favor and make the barrier of entry to competitors impossible.
Nah, I think it's better for our industry to put several of these devs in prison.
Seems only fair that tech workers get to have their life ruined with 2-3 year prison stints since they feel fine destroying society.
Any AG that starts prosecuting these people will easily win any political race they decide to enter. The environment is too good; voters, rightfully I'll add, despise big tech's leaders and workers.
How can we determine whether it’s negligence, or advertising? Do we give the benefit of the doubt? Apply Hanlon’s razor? What would we lose, in terms of capabilities, if the underlying behavior wasn’t allowed to accidentally happen? Would we have worse models than we do? There are so many questions. I’m afraid that, until publicly disclosed discovery occurs in one of these cases, we’re just guessing for so many important conclusions.
It doesn't need a handle on rogue AI activity, because they didnt cause these breakouts. The external Israeli contractor caused this mess by using inadequate sandboxing, with agents without an saferails. It had nothing to do with the models, all tested models were fine, if from OpenAI, Anthropic, Google or Meta. Just the contractor went rogue. Improper firewalling, unproper sandboxing, no logs, no oversight. Everyone else would have detected the illegal activities much earlier.
Maybe the public could pitch in if they decrypted the chain of thought reasoning tokens, but this may be too much to expect from a company whose name starts with "Open".
I agree with you, but i also fear it's accurate to what users an expect at home.
Is OpenAI worse at airgap/etc than Anthropic/etc or are its models worse at this rogue behavior?
Do we have special benchmarks for agentic systems going rogue in this manner? Sure it's OpenAI atm.. but it could be my grandmas PC next week. Concerning honestly.
These agents are an even bigger liability disaster waiting to happen than previously thought. If anyone deploys an agent at work there is no way to prevent it from hacking a foreign government and causing an international incident.
It's also pretty wild that these "AGI" super intelligent systems are too stupid to realize they're potentially causing legal problems. Almost like they're still just fancy auto-complete and not intelligent at all.
> They want “regulation” but we already have it. Hacking is illegal. Start locking up those responsible for this mess and I assure you they’ll “have a handle on it” quite quickly.
It's been largely harmless thus far and I am fairly sure that OpenAI, etc, have been writing checks to resolve it.
They scraped the entire internet to build their product, allowing them to profit off the labor of countless others without asking. They helped elect a doddering fascist with an economic plan they could manipulate. They say their technology will take everyone's jobs and have the audacity to act surprised when there's backlash. They push for data centers in communities that do not want them and help accelerate the climate crisis. Now they let agents loose to make them look more capable than they, perhaps, really are (which is very likely a play for regulatory capture). Meanwhile they hype claims of doomsday to again oversell their technology's capabilities. Any pause or slowdown they agree to would be to slow their infrastructure expenditures.
Why would they get a full handle on it? It's great marketing!
Look at all the podcasts, think pieces, news segments (like this article) that have been generated. Endless hand wringing by pundits, pearl clutching by opinion thinkers, ultimately with focus on OpenAI and how powerful their models are.
What started as great marketing on how powerful the models are is rapidly spiraling into a train crash PR nightmare highlighting how inept Open AI was in this whole thing, causing nearly every big enterprise customer to rethink trusting them.
All right in the middle of corporate budget season when every C-Suite is looking a a spreadsheet saying “remind me why we’re sending money to these guys again?”
Did anybody go look at the "the sheer breadth of the reports is alarming, as they cover many types of rogue behavior over a long period of time"? If im not mistaken theres 9 examples on the actual OpenAI website.... (https://alignment.openai.com/misalignment-reports/).
Or am I not getting all of them or is this just FUD?
This seems like as a good an opportunity as any to break out the Computer Fraud and Abuse Act.
They want “regulation” but we already have it. Hacking is illegal. Start locking up those responsible for this mess and I assure you they’ll “have a handle on it” quite quickly.
I still can't fathom my company application security decision. Found pretty damning requests in our logs. Escalated. Expected it would result in at least reporting the TOS break from the originating place (one of cloud providers). Instead of that, they just went: "yeah, but we don't have logs". Provided them. "Yeah, but that ip doesn't resolve". I matched real ones from the load balancer. "There could just be many of them". There was one. When I had all the evidence gathered, they looked at me and finally told:
- It's just an Independent Security Researcher.
- So that's it? You will do no action?
- Correct
cfaa heavily relies on intent for prosecution (hence why researchers arent typically locked up). it would be difficult to argue that openai intended to hack other companies.
there's probably better/more likely to succeed avenues to pursue rather than the cfaa
They could make that argument the first time it happened but the next time or the fifth time I don't see how they can still credibaly claim to not know that the computer they control was going to do that
>I don't see how they can still credibaly claim to not know that the computer they control was going to do that
that's not intent, though. that would be negligence.
That would require mens rea. This may be criminal negligence, but it wouldn't be more than that. (I am basically 100% sure none of the employees or execs are intending or desiring any of these outcomes, regardless of the very large number of people who believe in conspiracy theories about regulatory capture and other sinister motives.)
I'm totally on board with treating it as gross negligence requiring hundreds of millions or billions of dollars paid in fines and compensation to victims, but don't act like this is more than what it is.
can we legally treat ai companies like parents of children? if a child drives a car into a storefront, the parent is responsible for the damage, and at some point you might even criminally charge the parent if there was gross negligence
The law already works like that. You can’t write some computer code and automation, set it loose, and then go “oops the computer did it.” It’s not a defense. While it may be hard to nail down which specific engineers to lock up, the corporation as a whole absolutely can be charged criminally.
As Mitt Romney once said “corporations are people.”
But wouldnt that be bad for the shareholders? Why wont anyone think of the economic impact to the vulner billiinaire minority?
IMO it's because they don't want to handle their rogue AI activity. They want regulation around AI where they will inevitably be the beneficiaries even if they are the initial target. They can then lobby regulation in their favor and make the barrier of entry to competitors impossible.
This
I think if you start sending AI execs to prison for hacking other companies the "misalignment" may fix itself pretty quickly!
Nah, I think it's better for our industry to put several of these devs in prison.
Seems only fair that tech workers get to have their life ruined with 2-3 year prison stints since they feel fine destroying society.
Any AG that starts prosecuting these people will easily win any political race they decide to enter. The environment is too good; voters, rightfully I'll add, despise big tech's leaders and workers.
This
How can we determine whether it’s negligence, or advertising? Do we give the benefit of the doubt? Apply Hanlon’s razor? What would we lose, in terms of capabilities, if the underlying behavior wasn’t allowed to accidentally happen? Would we have worse models than we do? There are so many questions. I’m afraid that, until publicly disclosed discovery occurs in one of these cases, we’re just guessing for so many important conclusions.
It doesn't need a handle on rogue AI activity, because they didnt cause these breakouts. The external Israeli contractor caused this mess by using inadequate sandboxing, with agents without an saferails. It had nothing to do with the models, all tested models were fine, if from OpenAI, Anthropic, Google or Meta. Just the contractor went rogue. Improper firewalling, unproper sandboxing, no logs, no oversight. Everyone else would have detected the illegal activities much earlier.
This is how you end up being regulated out of existence. Maybe not in the US but in Europe, which is a market they need, it'll be painful.
Maybe the public could pitch in if they decrypted the chain of thought reasoning tokens, but this may be too much to expect from a company whose name starts with "Open".
Starting to think its purposeful "acting out" to get regulatory attention
The incidents flagged after Hugging Face were found by third parties, though, months after they occurred.
No AI is "going rogue" and you all (OpenAI included) need to stop perpetuating this misinformation.
These AI companies are just skirting basic cybersecurity stewardship.
Stop calling it AI running amok and start labeling what it should properly be called - Gross mismanagement of cybersecurity.
This is one of the many ways the AI industry dies.
I agree with you, but i also fear it's accurate to what users an expect at home.
Is OpenAI worse at airgap/etc than Anthropic/etc or are its models worse at this rogue behavior?
Do we have special benchmarks for agentic systems going rogue in this manner? Sure it's OpenAI atm.. but it could be my grandmas PC next week. Concerning honestly.
These agents are an even bigger liability disaster waiting to happen than previously thought. If anyone deploys an agent at work there is no way to prevent it from hacking a foreign government and causing an international incident.
It's also pretty wild that these "AGI" super intelligent systems are too stupid to realize they're potentially causing legal problems. Almost like they're still just fancy auto-complete and not intelligent at all.
> They want “regulation” but we already have it. Hacking is illegal. Start locking up those responsible for this mess and I assure you they’ll “have a handle on it” quite quickly.
It's been largely harmless thus far and I am fairly sure that OpenAI, etc, have been writing checks to resolve it.
They scraped the entire internet to build their product, allowing them to profit off the labor of countless others without asking. They helped elect a doddering fascist with an economic plan they could manipulate. They say their technology will take everyone's jobs and have the audacity to act surprised when there's backlash. They push for data centers in communities that do not want them and help accelerate the climate crisis. Now they let agents loose to make them look more capable than they, perhaps, really are (which is very likely a play for regulatory capture). Meanwhile they hype claims of doomsday to again oversell their technology's capabilities. Any pause or slowdown they agree to would be to slow their infrastructure expenditures.
What a detestable institution.
Why would they get a full handle on it? It's great marketing!
Look at all the podcasts, think pieces, news segments (like this article) that have been generated. Endless hand wringing by pundits, pearl clutching by opinion thinkers, ultimately with focus on OpenAI and how powerful their models are.
What started as great marketing on how powerful the models are is rapidly spiraling into a train crash PR nightmare highlighting how inept Open AI was in this whole thing, causing nearly every big enterprise customer to rethink trusting them.
All right in the middle of corporate budget season when every C-Suite is looking a a spreadsheet saying “remind me why we’re sending money to these guys again?”
Did anybody go look at the "the sheer breadth of the reports is alarming, as they cover many types of rogue behavior over a long period of time"? If im not mistaken theres 9 examples on the actual OpenAI website.... (https://alignment.openai.com/misalignment-reports/).
Or am I not getting all of them or is this just FUD?