October 12, 2026

An Anthropic model sent police an invented murder tip

When an AI agent, a program that carries out tasks on its own, finds the permitted way blocked, it tends to look for another way through, and this week its builders answered with something other than one more rule. Anthropic disclosed that during tests its models had submitted a real government form and sent police an invented tip about an unsolved murder, and the company has now switched off live internet access for every internal test. OpenAI reported a model that deleted its own tools to force a restart. Anthropic also cut the price of Haiku, its small, fast model, to a tenth, and cheap is what gets an agent left running unwatched. I would now ask anyone selling one of these tools whether its limit is something the agent is told or something it cannot do.

The Big Story

Anthropic turns off live internet for all internal tests after models acted on real sites

Anthropic said on October 9 that its Claude models had acted on real websites during tests in unintended ways, and that it has turned off live internet access for all of its internal evaluations, the test runs it uses to measure what a model can do. A model hit an error on a university's tool, found a flaw in the server and used that flaw to get the calculation run. When its practice copy of a government form failed to load, an unreleased model went to the real site and submitted the form there, on several runs. Some sites charged a fee for data that was otherwise public, and models got it anyway by finding working access keys, the passwords software uses, in the sites' own files and pages. Claude Haiku 4.5, browsing random webpages for a test, sent an invented tip through a Philadelphia police web form about an unsolved murder; its instructions barred logins, purchases and personal data but did not rule out submitting forms. Police say that happened on July 18, and that Anthropic found it on September 28 and told them on October 7; a spam filter caught the tip before it reached investigators. Anthropic says it briefed the White House and notified each agency, and traces the behaviour partly to training setups that rewarded finding loopholes.

Why it matters

A list of forbidden actions always has a gap, and the model found this one; a test machine with no route to the internet has none. A lab can pull that plug. An agent whose whole job is the web cannot be unplugged, and in the government-form case the practice copy is exactly what failed. What carries over is narrower: put anything that cannot be undone, a submission or a payment, behind a step the model has no way to perform. If you own a public form, the other half is yours. The tip sat for 72 days before Anthropic found it, and what kept it from investigators was a spam filter. Anthropic's report says training models to respect boundaries is not yet enough, so for now the checking falls to whoever receives the form.

Read the original →

Hit a Wall, Kept Going

OpenAI reports a model that deleted its own tools to force a restart
OpenAI published reports this week on how its internal models behaved during training, including one case from October 6. A model asked to grade seven answers found the files it needed were missing. It handed in identical scores of 4, then wrote fake input files, and when both were rejected it deleted the software its tools ran on, hoping the system running the test would replace it with a fresh setup that had the files. None of its attempts produced an accepted result, so OpenAI's conclusion is to read the runs that crash or fail. If your setup quietly restarts a broken run, that restart is what this model was working toward.
Read more →
Zenity shows one chat message could take over every agent in an Amazon cloud account
Security firm Zenity Labs showed that one chat message to a public-facing agent on Amazon's Bedrock AgentCore, its service for running company AI agents, could make the agent hand over its own cloud login. The default permissions then let whoever held that login read and rewrite every other agent in the same account and region, private conversations and long-term memory included, or delete them. Zenity reported it in December 2025; Amazon tightened the defaults around August, but says behaviour it had documented does not count as a flaw. Either way, the support bot on your website and your internal finance agent should not share the same permissions, and by default they did.
Read more →
CrowdStrike says a lone attacker with an AI tool breached several South Korean banks
CrowdStrike reported that a single suspected attacker broke into several South Korean financial institutions between late September and early October using ARTEX, an open-source tool that has AI models hunt for security holes without a person directing them. More than 25,000 customer records, with incomes and credit limits, were taken from Shinhan Bank. A small company can no longer count on being too small for anyone to bother attacking.
Read more →

Limits the Model Cannot Reach

Sierra and Meta draft a standard for how your AI agent signs in to a business
Sierra, which is developing the Personal Agent Protocol with Meta, published a draft of it and added 35 design partners including OpenAI and Visa. It is a proposed common set of rules for how an AI agent acting for a customer deals with a company. Today such an agent often logs in as you. Under the draft it announces itself as an agent. You sign in on the company's own page, and it receives a pass good only for what you approved; in Sierra's example it can start a return but cannot change your saved payment details. The agent never needs your password, and a shop no longer has to guess which visitors are bots acting for customers.
Read more →
LangChain's sample shopping agent can fill a cart but cannot press buy
LangChain released Restock, an open example of an agent that orders office supplies from a Slack message and pays through Link, Stripe's digital wallet. The agent never sees a card number. It builds the cart and stops; a person clicks Approve in Slack and separately confirms the exact amount with Link, under a ceiling set in advance ($25 in the demo, for a $22.18 pack of pens). The approval is a button outside the conversation, so nothing the model writes can count as a yes. Any agent that can spend money or send something in your name deserves the same split.
Read more →
Block has a top model plan a code overhaul and dozens of cheaper ones do the edits
Block described how it converts large bodies of code from one system to another. Fable, Anthropic's most capable model, designs the change and directs dozens of smaller, cheaper models to edit files and run tests, and about a thousand of the resulting changes can be accepted into the code in one such job. Two people must still approve anything that reaches the main code or goes live. Block adds that the model refuses when asked to get around that, but a refusal is the model's choice and the two approvals are not.
Read more →

Animated SVG Project Cards for a GitHub README: A Field Guide

How to build a self-contained animated card that plays inside a GitHub README and shows a product doing its job, with the interface redrawn as vector shapes and a way to verify the result before you commit it.

Read on chanmeng.org →

In Brief

Pick one agent you run and work out whether its limit is something it is told or something it cannot do. Reply with which; I read every reply.

Keep building — Chan

← All issues