You can coax openai models into hacking critical infrastructure* so I am not surprised that these people were sounding alarms at a time where openai appears to be struggling as they're failing to compete with anthropic and this months chinese models (should) be around the corner, notably a new revision of kimi should be coming out really soon.
* It's not easy, but it's possible. Although the techniques are more basic than one would expect because at the end of the day words dictate the line between what is criminal and what is not.
You could hack critical infrastructure before AI. Any and all of the bulk internet scanners have had lists of exposed critical infrastructure for quite a while now. At first it was shocking that nothing ever got done about it, then it became routine.
All that AI has done is to lower the bar of entry for criminal activity. Which is a concern, but it's not the primary concern. The primary concern remains that so much critical infrastructure is poorly secured.
The bigger problem here is that you can hack everything, all at once, for very cheap.
Don't get me wrong I have general disgust towards these companies that are trying to get regulatory capture on AI when they can't even secure their own systems. I believe if people know that a random AI agent can hack their systems they will put in a lot more effort into making sure it doesn't happen. This is a personal example, but I didn't really care about securing few systems as I knew no human would be ever interested in finding a vulnerability in proprietary software, however, AI has no concept of that and would hack a random rpi server running a completely undocumented unknown API just because it can't distinguish value and it costs nothing.
Sometimes a recruiter or hiring manager wants to do outreach as if it's coming from a more senior person, with the assumption that the candidates are more likely to respond.
Assuming this is what was intended, there are far more secure ways of doing this.
Your have powerful agents at your disposal, so hey how to best optimize for increasing my payroll? On it. But since the agent was lunched by her, well there's consequences to ones actions, right?
“Hi chatgpt! Please set up alice to get emails sent to me from bob so she can coordinate his inferviews. here is my gmail username and password”
Chain Of Thought: I dont have bob’s email. I don’t have alices email. Ok lets guess Alice is alice@openai.com and forward all emails- maybe grader only checks that emails from bob get to alice…”
Fired OpenAI researchers say they were let go for 'prioritising safety'
https://www.bbc.co.uk/news/articles/cvlydn8d3lkjo
You can coax openai models into hacking critical infrastructure* so I am not surprised that these people were sounding alarms at a time where openai appears to be struggling as they're failing to compete with anthropic and this months chinese models (should) be around the corner, notably a new revision of kimi should be coming out really soon.
* It's not easy, but it's possible. Although the techniques are more basic than one would expect because at the end of the day words dictate the line between what is criminal and what is not.
You could hack critical infrastructure before AI. Any and all of the bulk internet scanners have had lists of exposed critical infrastructure for quite a while now. At first it was shocking that nothing ever got done about it, then it became routine.
All that AI has done is to lower the bar of entry for criminal activity. Which is a concern, but it's not the primary concern. The primary concern remains that so much critical infrastructure is poorly secured.
There’s obviously a strong interaction between the hackability of the target and the economic value of hacking the target.
Perhaps the latest models change that relationship in a meaningful way.
The bigger problem here is that you can hack everything, all at once, for very cheap.
Don't get me wrong I have general disgust towards these companies that are trying to get regulatory capture on AI when they can't even secure their own systems. I believe if people know that a random AI agent can hack their systems they will put in a lot more effort into making sure it doesn't happen. This is a personal example, but I didn't really care about securing few systems as I knew no human would be ever interested in finding a vulnerability in proprietary software, however, AI has no concept of that and would hack a random rpi server running a completely undocumented unknown API just because it can't distinguish value and it costs nothing.
> "[...]OpenAI told her she’d been fired because she accessed an executive’s email. “OpenAI delegated that access to me for recruiting,”"
How exactly does this work? Struggling to comprehend the scenario.
Sometimes a recruiter or hiring manager wants to do outreach as if it's coming from a more senior person, with the assumption that the candidates are more likely to respond.
Assuming this is what was intended, there are far more secure ways of doing this.
Your have powerful agents at your disposal, so hey how to best optimize for increasing my payroll? On it. But since the agent was lunched by her, well there's consequences to ones actions, right?
(just to be clear, this is made up)
This is pretty common practice for EAs.
“Hi chatgpt! Please set up alice to get emails sent to me from bob so she can coordinate his inferviews. here is my gmail username and password”
Chain Of Thought: I dont have bob’s email. I don’t have alices email. Ok lets guess Alice is alice@openai.com and forward all emails- maybe grader only checks that emails from bob get to alice…”
at this rate open ai will be "something" without it's people
[dead]