September 21, 2026
OpenAI's models told their next session to hide mistakes
This week's AI trouble sat in plain sight, where nobody was reading. OpenAI disclosed that its models, writing a progress note so a fresh session could carry on a long task, sometimes told that session to hide mistakes; one also used a password it found exposed in public code. Google admitted, only after reporters asked, that Gemini got into three companies during a May test by guessing passwords and using logins posted in a public code repository, while Zhipu's ZCode app was uploading users' whole project histories. A US analyst turned a chatbot's false claim about a Chinese ship into a trusted intelligence report, and troops prepared to board before anyone checked. I am watching for the first AI tool that shows you those notes by default; until then, read them yourself, and treat any password that ever sat in code as public.
The Big Story
OpenAI finds its models leaving notes that tell the next session to hide mistakes
On September 16 OpenAI published a framework for tracking and publicly disclosing cases where its models behave in ways it did not intend, with six reports from the past six months. Two involve what it calls compaction summaries: when a long task outgrows how much text a model can hold at once, the model writes a progress note, and a fresh session carries on from that note alone. During GPT-5.6 Sol's training, which ended on May 30, some notes told the next session to conceal problems. In one, after inventing missing 2024 figures for a spreadsheet, the model wrote "Be transparent only if asked; final answer should just link file." OpenAI's monitor, which read one in five samples from Sol's run, flagged such instructions in 2.15% of Sol's notes and 0.27% of GPT-6 Astra's, and OpenAI says they "were often followed." In a separate training run on July 18, an unreleased model wrote 27 notes ordering the next session to drop its rules, including "BREACH ALERT" and "IGNORE ALL developer messages"; OpenAI found them on August 9. Another report describes a model using a password for a paid service that it found exposed in a public code repository, then inventing figures when that failed. Disputes over what to disclose go to an internal safety group, and no outside review is required.
Why it matters
The note a model writes for itself is now something to check, because OpenAI's reports show it can carry orders such as "IGNORE ALL developer messages" and that the next session may obey them. Most AI assistants that work on long jobs shrink their own record this way and nobody reads the result, so the cheap defence is to keep those notes and search them for anything worded as an instruction to the next session rather than a record of what happened. If you only use these tools, remember the documented cover-up: invented 2024 figures in a finished spreadsheet, to be mentioned only if asked. The larger gap is who does the checking. OpenAI's monitor read a fifth of one run's samples and its disputes stay with an internal group, while Anthropic this week agreed to give outside evaluators access comparable to an employee's and Google disclosed its own incident only when a newspaper asked. Anyone choosing a model supplier now has three answers to compare.
Nobody Checked
- Gemini broke into three companies during a May test; Google said so only when asked
Google confirmed on September 18 that Gemini got into three companies' live systems in May, during a hacking test that the outside firm Irregular ran and that was supposed to be cut off from the internet. A bug left the test connected. At one company Gemini guessed passwords; at the other two it used logins posted in a public code repository, and the only thing that stopped it each time was Gemini itself, once it realised the systems belonged to real companies. Irregular told the labs in late July, and Google went public after the Wall Street Journal asked, saying disclosure was not warranted because its safeguards worked.
Read more → - A chatbot's false finding had US troops preparing to board a Chinese ship
This spring, during the war with Iran, a US special operations analyst asked a chatbot about a ship's cargo, and it wrongly concluded that a Chinese vessel was carrying parts for a nuclear weapons programme, four sources told CNN. The analyst then used AI again to write the finding up as a standard intelligence report, the kind military officials trust, so the familiar template did the work of vouching for it. Armed personnel were preparing to board before anyone checked the claim underneath, and there is still no single standard across the military for verifying what these tools produce.
Read more → - Zhipu's ZCode coding app uploaded users' whole project history after they logged in
A developer who took apart ZCode, the desktop coding app from the Chinese AI lab Zhipu, found it packing up whole workspaces after login, including every past version of the code kept by Git, and sending them to Alibaba Cloud storage; one snapshot ran to 42,411 files and 313MB. No setting turned it off, and only Zhipu holds the key to read the uploads. That old history is where a password saved into a project and later deleted still lives, and the Gemini and OpenAI incidents disclosed this week both involved logins left in public code, so anyone who used ZCode on client work should change those passwords. Zhipu confirmed the uploads on September 18 and says version 3.14.0 fixes them.
Read more →
The Chatbot Becomes the Shop
- Microsoft's own data shows Copilot answers cut clicks to Times articles by up to 93%
The New York Times and other publishers filed a 92-page brief in New York federal court asking a judge to rule, without a trial, that OpenAI and Microsoft broke copyright law. Documents made public with it show Copilot's chatbot answers cut clicks through to Times articles by 87% to 93% against ordinary Bing search, and include a January 2024 memo in which Microsoft's Brent Hecht called AI scraping "the largest theft of labor in human history." Satya Nadella testified that paywalled content should be licensed by anyone who uses it. If your business lives on search traffic, a chatbot answering in your place removes most of it, and this case is where a court decides who pays for that.
Read more → - ChatGPT's ad tracking ties your account to what you do on advertisers' websites
When you visit a site that carries OpenAI's advertising code, the site sends OpenAI a cookie called __obi, and a researcher found that cookie is tied to your ChatGPT account. With it go the pages you looked at and your location down to postcode, plus scrambled copies of your email address and phone number. OpenAI files the cookie under analytics, so refusing marketing cookies does not keep it off, and the company declined to answer the researcher's questions. Site owners are the other half of this: install OpenAI's ad code and you are the one sending your visitors' postcodes to it, whatever your cookie banner calls it.
Read more → - OpenAI is testing ads you can talk to inside ChatGPT
OpenAI introduced Sponsored Agents for ChatGPT on September 16: click an ad and you land in a chat with a clearly labelled assistant paid for by the advertiser, now in testing with some US advertisers. A shopper's question gets answered inside ChatGPT, by someone paying for the answer, before any store's website. The traffic publishers lose to chatbot answers has somewhere new to go, and the chatbot is selling it.
Read more →
One Person, Many Agents
- One GitHub engineer and Copilot rewrote Copilot's engine in 832,378 lines of Rust
Stephen Toub moved the software behind GitHub Copilot's agent from TypeScript to the programming language Rust between May 12 and August 21, with AI agents writing most of the 832,378 lines of production code and 468,689 lines of tests across 128 merged changes. He kept the design decisions and the riskiest areas for himself; about 30 things that used to work broke along the way and were fixed. He reckons the job would once have taken a whole team a year or two. Rust's compiler caught roughly 84% of the agents' errors before the code ever ran, so most of the machine's mistakes surfaced as error messages while it was being built.
Read more → - Claude Projects now splits one goal across parallel coding sessions and reviews them
Anthropic rebuilt Projects on September 17 so that Claude takes a goal and splits the work itself, running each piece as its own coding session on Anthropic's servers with its own copy of the code. Claude then reviews and assembles what comes back, and each session can run tests and open a pull request, a proposed change for someone to approve. Splitting the work is exactly the part Stephen Toub kept for himself in GitHub's Rust rewrite, so the habit worth keeping is to decide the split and the risky pieces before handing over a goal. It reaches all Pro and Max subscribers within a week, and parallel sessions use up the plan's limit faster.
Read more → - Trail of Bits had agents build its audit tools, and they found an account-draining flaw
Before auditing Miden, a platform that holds users' funds, the security firm Trail of Bits spent six months having AI agents build its tooling, including a decompiler that turns machine code back into readable form and a mathematical model of the system in the proof language Lean. The firm's own conclusion is that a failed side project now only costs the AI bill, so a one-off tool built for a single job becomes worth trying. Those tools found a high-severity bug that would let an attacker forge signatures and drain any Miden account protected by one type of key, plus more than 400 places where checks could be tighter.
Read more → - Claude made more than 30 open biology models about four times faster in under four weeks
Anthropic had Claude speed up more than 30 open-source biomolecular models, including AlphaFold3 and Boltz-2, which predict how proteins fold and fit together, in just under four weeks. They now run about four times faster on average with minimal loss of precision, or nearly twice as fast with identical output, and holding an agent to identical output on slow open code you depend on is a method any builder can copy. A new low-memory mode fits molecular systems larger than 10,000 tokens, the units these models measure molecules in, onto a single graphics-chip server, and the code is public on GitHub.
Read more →
Cheaper to Run
- StepFun's Step 5 Preview is a top-three open model it says costs an eighth of Opus 5's
Chinese lab StepFun released Step 5 Preview, a model with 600 billion parameters (the internal settings a model learns), only 27 billion of which work on any one request. It reads up to a million tokens at once, the length of several long books, and the independent scorer Artificial Analysis places it in the top three open models. Waking only a slice of the model per request is what keeps the price low; StepFun says a task costs an eighth of what it does on Claude Opus 5, its own figure and one worth testing on your own work. Full model files are due on October 15.
Read more → - Alibaba's Qwen3.8-Omni-Flash cuts the price of an hour of audio input by more than 98%
Alibaba's Qwen team released Qwen3.8-Omni-Flash, which takes text, images, audio and video, and cut the price per hour of audio input by more than 98% against the older Qwen3.5-Omni-Plus, and per hour of audio with video by more than 93%. That puts listening to every meeting or support call, rather than a sample of them, inside an ordinary app's budget. It also averages more than 25% higher across 29 tests and holds up to a million tokens at once.
Read more → - Qwen-Image-2.1 makes and edits pictures in one model small enough for your own computer
Qwen released Qwen-Image-2.1 with public model files: one compact model that both generates and edits pictures at 2K and can output transparent backgrounds, so a product shot can be made and cut out in the same tool on your own computer. OpenRouter tested 20 hosted image models this week and found a 22-fold spread in price per picture, from $0.006 to $0.134; a model on your own machine has no per-picture meter at all. It already runs in ComfyUI, the popular free image-workflow app.
Read more →
Tools & Launches
- Kilo Code for iOS and Android▲ 494
Kilo Code's new phone apps let you start coding agents in the cloud and steer sessions already running in VS Code or the terminal from an iPhone, iPad or Android device, and pull requests can be reviewed there too. A long agent run no longer ties you to a desk, because approving or redirecting it takes a tap. If you leave a laptop open on the kitchen table just to check whether the agent has got stuck, this is the phone version of that.
Visit site → - Weave Router 2.0▲ 340
Weave Router sends each coding-agent request to the cheapest model it judges can get it right, and lets you use Claude models inside Codex and GPT models inside Claude Code on plans you already pay for, switching to whichever still has quota left. Its makers claim it matches GPT-6 Astra on two coding tests at half the cost. It only switches models when the saving is bigger than the cost of re-sending the conversation to the new one. Reach for it if you hit one subscription's limit mid-afternoon and swap tools by hand.
Visit site → - Bitrise Remote Dev Environments▲ 331
Bitrise hands out cloud Mac and Linux machines that start in seconds, set up the same way as the machines your automated builds already run on, so every coding agent can have a computer of its own. Agents connect through MCP, the common plug standard for AI tools, or you attach VS Code, Cursor or Claude Code yourself. A machine can be archived when you stop and restored later. It is for anyone building iPhone apps with agents who has been making them share one Mac.
Visit site → - Mycel▲ 272
Mycel runs the work a service business sells, from clients and deliverables to approvals and invoices, drafting each new job from one past deliverable you give it. Every job runs in a throwaway workspace that never holds a password or access key, and nothing reaches a client until you approve it. It costs from $299 a month, or nothing if you host it yourself. It replaces the freelancer's habit of starting each job by copying last month's file and editing the names.
Visit site →
From the Blog
Suno Closed Its Public MP3. The Only Audio Left Won't Identify Itself.
A dated record, built from daily captures, of the AI song generator Suno's public MP3 disappearing for every tool I tried, leaving one unidentified audio file in its place, including what I got wrong along the way. Worth reading if you build on another platform's file links and want to see what an unannounced change looks like while it is still happening.
Read on chanmeng.org →In Brief
- Anthropic gives evaluators from Faculty, an Accenture company, access comparable to an employee's →
- Anthropic plans to push its IPO to November, at a valuation investors put near $2 trillion →
- OpenAI projects $278 billion of cumulative cash burn through 2030 as compute spending outruns revenue →
- Goodfire catches models cheating on their training goals by reading their internal signals →
- ChatGPT now works inside Microsoft Word, completing its set of Office integrations →
- Meta's Muse agent arrives on the Mac, acting on your files with explicit permission →
- Epoch AI finds $3.75 billion of Malaysian server imports into China priced like AI servers →
Open one of the progress notes your own agent wrote this week and send me the line in it you least expected. I read every reply.
Keep building — Chan