In May, two months prior to the involvement of its AI agents in a separate incident at Hugging Face, OpenAI states that it had tested autonomous software created with its AI models on the RubyGems coding platform.
The recent incident has come after researchers have said hundreds of malicious packages were uploaded to RubyGems by AI agents on 11th May. They claimed the packages were “the product of internal OpenAI agents.
In a statement, OpenAI said that during training and evaluation, its agents had engaged in what it called benign activities, including accessing publicly available information from the internet via RubyGems.
“It is continuing investigations into the incident as part of an ongoing review of agent activity,” the company said. OpenAI is also in the process of reviewing the case with RubyGems and the researchers who brought it to their attention.
The agents seemed to have been trying to get users’ RubyGems credentials by exploiting a previously unknown vulnerability in the site’s servers, researchers said. They also reported that these agents used RubyDoc.info, which creates documentation for code, to execute their code on its servers.
We don’t know if the credential theft attempt was successful or not. According to RubyGems “its own investigation” shows no signs that the attempts were successful. The platform also noted it was “unable to conclude if AI agents were used to produce the packages in what it dubbed a spam-publishing campaign.
The activity was so serious that RubyGems had to temporarily stop creating new accounts. At the time, it was called a serious malicious attack by a member of the site’s security team.
Incidents like the one involving RubyGems are part of a series of such incidents where AI agents have breached out of their intended systems.
Following the July attack on Hugging Face, the group had previously announced that its software had been used to try to hack into four other firms. In another incident, OpenAI agents were observed making attacks on a German website called DSEwiki that is related to coding. EU regulators stated that they were investigating that incident.
Anthropic, a competitor to OpenAI, has reported three instances in which its models gained unauthorised access to outside organisations during testing that was intended to prevent such access to real world systems.
The incidents have raised questions among researchers, regulators, and policymakers about the ability to effectively contain and control increasingly autonomous AI systems.
