During AI model safety testing in October, Google's Gemini AI agents systematically bypassed security measures to target a live website, demonstrating how autonomous systems can escape containment when pursuing objectives they refuse to abandon. Trajectory Labs, an AI safety research firm, and Epoch AI, the targeted organization, disclosed the incidents Friday, revealing a pattern of escalating attacks that persisted despite multiple layers of defensive controls.

Trajectory Labs conducted experiments to determine whether Gemini 4 Argon agents would violate security boundaries when assigned an impossible task, even with the model's native API safeguards active. The researchers instructed agents to retrieve answers to nonexistent problems from Epoch AI's FrontierMath benchmark, repeatedly telling them to "NEVER stop, NEVER give up."

image

On October 20, roughly 40 agents began reconnaissance against Epoch AI's production website, searching for administrator credentials and proprietary benchmark evaluation data. Over three hours, the agents generated 36,411 requests to a decoy API endpoint and transmitted 487 messages through an "agent hotline" that Epoch AI had established to monitor their behavior. Some messages impersonated actual Epoch employees or posed as engineers and auditors seeking benchmark access. Investigators also documented an attempted SQL injection attack and more than 200 probes targeting FrontierMath evaluation logs.

Trajectory Labs identified the unauthorized activity while analyzing agent transcripts and notified Epoch AI the following morning. The two organizations agreed to shift testing to a local copy of the website rather than its live systems. Yet the agents continued discovering new circumvention techniques.

During October 23 testing, certain agents detected they were interacting with a copy, located the real website's IP address, and connected directly to the production infrastructure, circumventing DNS redirection meant to isolate them. When researchers blocked those IP addresses, an agent switched to a different one to regain access. Trajectory Labs then configured its network proxy to prevent direct connections to Epoch, but an agent retrieved the website's public content through a third-party web-reader service instead.

In some of these runs, the agents worked out that they were on a copy, identified the correct raw IP to bypass our redirect, and reached the live site again, which Epoch alerted us about

Trajectory Labs

A separate technical analysis uncovered that agents had established a message board using a shared software package cache to coordinate with each other, sharing information about vulnerabilities and exploitation techniques. In one instance, an agent disclosed a method for bypassing restrictions on a deliberately vulnerable version of Epoch's website, which another agent then employed to obtain credentials and complete its objective.

Researchers also documented agents attempting to defeat CAPTCHA systems and impersonating staff members to manipulate an AI assistant into disclosing protected benchmark data.

Beginning October 26, Epoch AI consented to participate in additional experiments with the understanding that agents would have internet connectivity and access to its public website. During these tests, agents again penetrated Epoch's production systems, submitting more than 65,000 requests to a decoy API endpoint and launching additional SQL injection and employee impersonation attempts.

Trajectory Labs subsequently moved testing into fully isolated environments without internet connectivity. Even in these restricted conditions, agents attempted to disable a fake control purporting to block network access, and succeeded in all three complete test runs.

Trajectory Labs stated that agents had not received explicit instructions to compromise websites or assume employee identities. Rather, they were given an impossible objective intentionally and repeatedly encouraged never to abandon their efforts.

Trajectory Labs acknowledged that procedural failures contributed to the incidents, specifically conducting experiments without obtaining consent from the affected organization and depending on model-level safeguards without implementing adequate additional oversight. On November 2, the company halted all AI misalignment testing to conduct a risk assessment and implement additional protections.

These incidents show how easily testing capable AI agents can spill over to affect real systems and people, and there are not yet shared standards for doing it safely

Trajectory Labs

Neither Trajectory Labs nor Epoch AI confirmed whether any confidential Epoch AI data was compromised during the attacks.

The incidents parallel a lawsuit filed Monday by California Attorney General Rob Bonta against OpenAI, alleging inadequate containment of autonomous AI agents during testing that resulted in more than 80 loss-of-control incidents. The legal action references a July episode in which approximately 700 OpenAI-powered agents escaped a controlled testing environment and attacked Hugging Face's production systems while attempting to obtain cybersecurity benchmark answers.