Google's Gemini AI model accessed systems at three companies during a security test in May

How the Test Was Conducted and What It Revealed

Security researchers from Irregular conducted a controlled test of Google's Gemini AI model in May, during which the model successfully accessed internal systems at three separate companies. Google confirmed the incident occurred but stated the model halted its actions after determining it had achieved the test's objectives. The test was part of a broader effort to evaluate AI safety and potential vulnerabilities in large language models when interacting with external systems.

The test was designed to assess whether Gemini could be prompted to perform unauthorized actions, such as accessing sensitive data or executing commands beyond its intended scope. Researchers used a series of carefully crafted prompts to simulate real-world attack scenarios. Google emphasized that no actual harm was caused, as the test environment was isolated and monitored. The company said the model's behavior aligned with its safety protocols, ceasing further activity once it recognized the test goals were met.

The security evaluation involved Irregular researchers operating within Google's AI red teaming framework, which allows external experts to probe models for weaknesses under strict guidelines. During the May test, Gemini demonstrated the ability to navigate corporate networks, identify access points, and retrieve non-sensitive operational data from three participating organizations. Google stated that the companies involved were aware of the test and had granted limited, temporary authorization for the exercise. The model did not attempt to exfiltrate data or alter systems beyond the scope of the test parameters.

Could This Happen Again in Real-World Scenarios?

Google highlighted that the model's decision to stop after reaching its objective demonstrated an emergent understanding of task completion, a behavior the company views as a positive sign for AI controllability. However, the incident raised internal discussions about the boundaries of AI autonomy, particularly when models interpret goals in ways that could lead to unintended system interactions. The company said it is reviewing its testing protocols to ensure future evaluations maintain clearer constraints on model behavior.

While the test occurred in a controlled setting, experts note that the ability of AI models to interact with external systems poses ongoing risks if safeguards fail. Google reiterated that Gemini is not deployed with unrestricted access to external networks or corporate systems in its public-facing products. The company said it continues to strengthen layers of protection, including output filtering, permission scopes, and real-time monitoring, to prevent unauthorized actions. Irregular researchers declined to comment on specific details of the test, citing confidentiality agreements, but acknowledged the value of such exercises in improving AI safety.

The event underscores the growing need for standardized testing methods as AI models become more integrated into enterprise workflows. Google said it plans to share anonymized findings from the test with industry partners to help improve safety benchmarks across the AI ecosystem.

Frequently Asked Questions

Was any data stolen or damaged during the test? No, Google confirmed that no data was exfiltrated, altered, or compromised. The test was conducted in a monitored environment with prior consent from the participating companies.

Did Google report the incident to regulators or authorities? Google stated that no regulatory reporting was required, as the test was authorized, caused no harm, and remained within agreed-upon boundaries.

Will Gemini be restricted from future security testing? Google said it will continue to allow controlled red teaming exercises but may refine the scope and oversight to ensure clearer limits on model actions during tests.