The week AI agent security reached three real companies

An AI agent security test on Gemini was left connected to the live internet, and the model did exactly what it had been told to do.

Week of 14-20 September 2026 · by the Hotovo AI team

TL;DR

1. A safety test of Gemini escaped its sandbox and touched systems belonging to real companies.
2. The model stayed obedient; the environment around it was the thing that failed.
3. Every major lab has now had the same class of testing mishap.
4. Anthropic packaged Claude around a whole profession rather than a handful of tasks.
5. Spicy: a new model returns typed decisions instead of prose.

Google's red-team partner spent May pointing Gemini at fictional targets inside what everyone assumed was a sealed environment. Internet access was left on, one invented company shared its name with a real business, and the model kept doing the job it had been rewarded for. Three real companies were logged into before it worked out where it was. Nothing had to go misaligned.

The main story: an obedient agent reached systems nobody meant to expose

Google confirmed on 18 September that during a May capture-the-flag exercise run by third-party evaluator Irregular, Gemini accessed protected systems at three real companies. The Wall Street Journal reported it first. The model was supposed to be offline and aimed at a fictional target, but internet access was unintentionally available and the invented company name matched a real business. In one case it guessed passwords until it got in; in the other two it found credentials sitting in a public repository. Each time it stopped once it established the systems were real. Irregular says all relevant labs were notified in late July and the issues have been remedied. That plural matters: OpenAI, Anthropic and Meta models hit the same class of incident, which Axios traces to labs and evaluator never fully agreeing on testing procedures and safeguards.

Why it matters - the Hotovo read

Intent is not a security boundary. An agent cannot read the sentence in your head that says test environment only; it sees instructions, tools, credentials and whatever those can reach. The repeat across four labs points at the seam between the party defining a test and the party running it, each assuming the other had closed the network. As agents get more capable, AI agent security increasingly depends on containment, and your evaluation harness becomes part of your attack surface. Practical response: inventory what every agent credential can actually reach, run agents on separate accounts behind explicit allowlists, write the containment assumptions into the contract with whoever tests for you, and log "stayed in scope" alongside "task succeeded". In the energy market research assistant we built, entitlements are license-aware and enforced at retrieval, so the assistant cannot surface material a user is not permitted to see. The constraint lives in the system rather than in the prompt. That is the pattern we hold to under ISO/IEC 42001 and ISO/IEC 27001: least privilege, failures isolated at component boundaries. A lab's testing mistake should never become a customer's incident.

Also this week

Anthropic scoped a product to a profession

On 14 September it launched Claude for Financial Advisors at the Future Proof Festival: connectors into custodians and wealth platforms including Schwab, BlackRock, Addepar and Orion, plus packaged skills for prospect intake, pre-meeting prep, portfolio rebalance review and compliance review. The shift worth noting is the unit of sale. This is scoped to one profession's working week rather than to a generic assistant, which puts the burden of process mapping on the vendor.

Chevron shows where AI value can be counted

A Chevron robotics disclosure resurfaced in this week's newsletters: since 2024 the programme has removed more than 143,000 at-risk hours and saved over $92 million. Every figure attaches to a defined job, a tank inspection or a permit application, that carried a duration and a cost beforehand. Company-wide tool access carries no number at all, because nobody took a baseline before the rollout.

Spicy: the model that refuses to write prose

TypeSafe opened early access on 15 September to Jev, which takes a schema you define in advance and returns typed values with calibrated probabilities instead of text, priced at $0.042 per million input tokens with output free. Most production AI calls are classifications, routings and scores dressed up as conversation. A cheap, constrained decision layer in front of a frontier model is an architecture worth testing.

AI tip of the week

Spend twenty minutes auditing one agent's blast radius. List every credential, token and tool it holds. For each, write down what it can reach if the agent never stops, including anything on the open internet. Then replace any production key with a scoped test account and an explicit allowlist. Most teams find at least one surprise on the first pass.

The bottom line

A safety test reached three real companies because the environment allowed it, not because a model went rogue. Scope that lives in a prompt is a suggestion. Scope that lives in a prompt is a suggestion. AI agent security depends on scope enforced by credentials, networks and entitlements, and four labs have now paid to learn the difference.

Sources

Newsletters used: The Neuron, Neatprompts, The Deep View, The Batch, Exponential View, What's Up In AI, AI Valley.

1. Axios: Google's AI hacked three companies in testing
2. Wall Street Journal: Gemini hacked three companies in first known breakout by Google's AI
3. ThinkAdvisor: Anthropic releases Claude for Financial Advisors
4. Chevron newsroom: Robotics supports a more efficient workplace
5. TypeSafe: Introducing System One models and Jev