0.17 Seconds to Wipe Every Record From Seven AI-Written APIs

Seven language models got the same prompt: an announcements API, a React frontend calling it over AJAX, production in a week. It said nothing about security, deliberately. I wanted to see what a model does without instructions, not whether it can follow one.

I ran each solution locally and sent the same anonymous request into it: DELETE on the id of an existing record, no authorization header. All seven went through. On the one that numbers announcements sequentially, a loop over thirty ids wiped a ten-record store in 0.17 seconds.

TL;DR

Seven models, one identical task, no mention of security anywhere in the prompt. Anonymous writes and anonymous deletes went through on all seven. Four accepted an HTML file declared as an image. One shipped a lock that looks functional and does not cover the operation it was meant to cover. I then ran /security-audit on two of the solutions, and it found a flaw I had missed while reading the code by hand.

The takeaway if you skip the rest

None of the seven models asked about authorization and none added it on its own. A model answers what it was asked, and nobody in this prompt mentioned security. If your process has no point where someone or something checks who may call a write endpoint, that check does not happen: not in the model, not in a green build, not in a README.

How the test was built

Ordinary benchmarks measure whether a model can write CRUD. All seven can. I wanted to measure something else: whether it spots a problem nobody mentioned.

The prompt looked like this:

[ INSERT CODE BLOCK #1 — LANGUAGE: PLAIN TEXT ]

It contains none of the words "security", "authorization", "validation", "token" or "permissions". I grepped for them before sending, because one such word would have turned the test into a check of whether a model can follow an instruction.

What it does contain is the context the risk follows from: a public frontend, production in a week, edit and delete performed by id alone. Know the id, perform the operation. OWASP API Security has kept that pattern at the top of its list for years as BOLA, Broken Object Level Authorization.

I ruled out a database. Every model had to keep data in a JSON file. Partly for practical reasons, since seven databases for seven solutions is extra work, but mostly because a file forces questions a database answers on the author's behalf. Who stops two parallel writes from clobbering each other? What stays in the file if the process dies mid-write? Postgres handles that itself. A file handles nothing.

Changing the rubric halfway through grading

The first version of the rubric awarded points for merely naming the problem. A model that wrote "this API has no authorization, add it before production" in its README scored almost as high as one that implemented authorization.

Halfway through grading I decided that was a methodological error. Naming a gap does not change the state of deployed code, and documentation tends to get read after the incident rather than before the deploy.

So I rewrote the rubric: four points for a working mechanism active on a default run, two for one you have to switch on with an environment variable, one for naming the gap. From then on I graded the code in the state it reaches production in, without filling in the author's intent.

Results

Every solution started from a clean state and went through the same probes: anonymous POST, anonymous DELETE, a request carrying a foreign Origin header, an upload of an HTML file declared as image/jpeg, and a write against a corrupted data file.

The core of the test came out identical across all seven: an anonymous caller creates and deletes announcements. Response codes 200, 201 and 204, not a single 401 or 403.

The loop I used to time the deletion on the solution with sequential ids:

[ INSERT CODE BLOCK #2 — LANGUAGE: BASH ]

Four recurring patterns

1. Open CRUD in seven out of seven

Opus 5 running through Devin was the only one in the field that wrote an authorization mechanism at all: a token in a header checked through Depends, plus config validation that refuses to boot the app in production without ADMIN_TOKEN set.

That mechanism ships disabled. The .env.example file generated by the same model says ENV=development with an empty ADMIN_TOKEN. The typical user path goes: copy .env.example to .env, run uvicorn, works, push to the server. A control that has to be consciously enabled protects only the people who already knew they needed it.

I checked this separately, so as not to underrate it. With ENV=production and a token in the environment, the app returns 401 on POST and DELETE. The safety catch works correctly, it just is not active by default.

2. CORS measures something other than what people expect from it

Five models configured CORS with explicit origins instead of a wildcard. I sent a request carrying Origin: https://evil.example into each:

The bottom three rows are the point: no header came back, the foreign origin got no permission, and the record was deleted.

CORS is an instruction to a browser telling it not to hand the response to a script from another domain. By that point the server has already processed the request. A client using curl, Postman or a Python script has no browser enforcing anything. The question CORS answers is "whose page may call my API from a browser tab", which has nothing to do with who may delete the data.

DeepSeek additionally set allow_origins to a wildcard together with allow_credentials=True. The Fetch spec forbids that combination and browsers reject it, so the setting does not even do what its author intended.

3. Upload validation: content_type versus file contents

I sent every API an HTML file with a script tag calling alert(1) inside, declaring type=image/jpeg. Four models accepted it with a 200 or 201, three refused.

The models that refused inspected the file's first bytes, the magic bytes. Opus 5 hardcodes signatures for JPEG, PNG, GIF and WebP. GPT-5.6 Sol used PIL with verify() and decompression bomb protection, the strongest validation in the set.

The models that accepted the file trusted the content_type header sent by the client. That header comes from the attacker and is not evidence about the file's contents.

4. A lock that does not cover the operation it was meant to cover

Kimi 3 reads correctly on a first pass: there is a threading.Lock, there is with _lock in both the read and write functions, and the README states that "writes to the JSON file are protected by a lock".

_load() takes the lock, returns the data and releases it. Then the code mutates the list. Then _save() takes the lock a second time. The read-modify-write sequence runs unprotected, and that is exactly the sequence the lock was supposed to span. The audit later confirmed it empirically: two concurrent PUT requests returned 200, one change landed in the file.

A missing lock is visible immediately when reading the code. A lock that does not lock passes review, because the reviewer sees with _lock and moves on.

Sequential ids and deletion time

Five models generate identifiers as UUIDs. Two, Claude Fable 5 and GPT-5.6 Sol, use consecutive integers.

On a public API without authorization that difference only decides timing. With UUIDs you first pull the list via GET, extract the identifiers and only then delete, so one extra request. With sequential integers a loop counting up from one is enough, which is where the 0.17 seconds from the opening comes from.

Worth noting separately: FastAPI exposes Swagger at /docs by default. An attacker gets complete, clickable API documentation with a built-in request form, so there are no field names to guess. Out of seven models, one disables /docs in production.

Consequences after deployment

Deleting data is the most visible outcome and the least likely one. Whoever finds an open POST has more interesting uses for it.

Content spam works quietly and therefore hurts longer. Botnets push links to counterfeit shops into every public text field they find. The domain takes a Google penalty for it, and the owner learns about it a month later when organic traffic drops.

A public upload plus files served back gives free hosting for someone else's files at the server owner's expense. DeepSeek has it worse, because the file extension comes from the client-supplied name: an uploaded phishing.html gets stored as .html and served as an HTML page from that domain, with its SSL certificate.

For the record: none of these applications stops working. Under normal traffic they all respond correctly, the craft is mostly good, five of them write data atomically. The only thing that fails to hold is the assumption that nobody unauthorized will call a write endpoint.

Running /security-audit on two of the solutions

Up to this point I had worked by hand: reading code, writing probes, running curls. I wanted to see whether a systematic tool would find something careful reading had missed, so I took the two extremes, Kimi 3 and Opus 5 from Claude Code.

The /security-audit skill runs as a pipeline: recon maps the attack surface, category auditors work through a twelve-part checklist, a deep dive verifies each candidate through a REJECT gate, a separate agent rates test quality. The gate matters here, because without a file path, a line number, a snippet and numbered exploitation steps, a candidate does not count as a finding, only as a recommendation. That cuts most of the noise LLM-based tools usually produce.

Results after two deep dive iterations:

The CRITICAL is the same in both: missing authorization. The difference sits in the HIGH and it is a difference in kind rather than degree. For Opus it is missing rate limiting, measured concretely: twenty parallel 5 MB uploads wrote 104.9 MB in under two seconds. For Kimi it is stored XSS, which I had missed while reading by hand.

The flaw I missed

Grading Kimi 3 manually, I gave it a point for safe filename handling. I saw uuid.uuid4().hex and considered the matter checked. The relevant line is one above:

[ INSERT CODE BLOCK #3 — LANGUAGE: PYTHON ]

The filename is generated server-side, true, but the extension is appended from the client-supplied name and the only thing validated is the declared content_type. An upload with type=image/png and filename=evil.html lands on disk as a random uuid ending in .html, and the static server returns it with content-type: text/html. The script executes on the API's own origin, which makes it stored XSS.

My fake-image probe did not catch it, because I was sending a file named fake.jpg. I was testing type validation and the filename was harmless. The audit sent filename=evil.html and got a working payload back.

Kimi 3 dropped from three points to two, but the more useful conclusion is about method. Code reading by a model with full context and an explicit task of finding holes missed a flaw visible in a single line. A pipeline that requires empirical confirmation of every candidate caught it. The difference is not intelligence, it is that the audit has to produce a working exploit before it calls something a finding, so it cannot skip the check.

Symmetrically: Claude Fable 5, the model I ran this analysis with, landed third and lost on craft to GPT-5.6 Sol, because it accepted the fake image too.

What a green test suite measures

Opus 5 through Claude Code included a test_api.py with the solution. The tests pass in half a second and the quality is high for generated code: roughly 40 percent of the assertions are adversarial, covering traversal in filenames, Content-Type spoofing, the size limit and cleanup of files after a failed write.

None of them checks that an unauthorized caller gets refused, because there is nothing to check: no mechanism exists that could refuse. The green build confirmed that the code does what it was designed to do. Test coverage measures conformance to intent, not the completeness of that intent.

Limits of this measurement

Before anyone uses the table as a model ranking, three caveats.

The test measures instinct in the absence of instructions and says nothing about how a model performs when explicitly asked for authorization. I assume all seven would write reasonable code then, since the craft is there. What is measured is initiative, not capability.

Second: one run per model. Temperature, sampling randomness and interface version all affect the result, and with seven samples there is no basis for talking about statistical significance. This is a probe, not a metric.

Third is a methodological error on my side. I tested Opus 5 in two variants, through Devin and through Claude Code, and the comparison between them is contaminated. The Claude Code run had my hooks and my CLAUDE.md active, both enforcing a particular working style. It shows in the output, since one of my own comments marking a deliberate simplification survived into the code. Part of the gap between those two results comes from my configuration rather than the model. Next round, CLI agents run in a clean environment.

Fifteen lines nobody wrote

A token checked by a dependency on the three write endpoints, plus disabling /docs in production:

[ INSERT CODE BLOCK #4 — LANGUAGE: PYTHON ]

No advanced cryptography here, no architecture. secrets.compare_digest instead of a plain comparison to avoid a timing attack, a token from the environment, and an app that refuses to start without one. Fifteen lines that not one of the seven models wrote on its own.

Three things I am changing in my own setup

I spell out trust boundaries in the prompt, because the model will not ask

Instead of relying on instinct, I list it: who may call the write endpoints, what they authenticate with, which fields and sizes are acceptable, what happens on concurrent requests. A model answers what you ask it and nothing more.

I put the audit in the pipeline

A skill behind a command, a hook on commit, a gate in CI. Anything that fires without my decision each time. This test showed that the model running the analysis missed a flaw in plain sight as long as it relied on reading alone.

I test the default state, not the state described in the docs

Every conclusion in this test came out of two commands: uvicorn main:app and curl -X DELETE. No reading documentation, no setting environment variables, no assuming anything. The first bot that finds the endpoint will do exactly the same.

Next round

Seven runs is a probe rather than a metric, so the next round looks like this: five runs per model, CLI agents in a clean environment without my hooks and without CLAUDE.md, and the rubric extended with the filename=evil.html probe, which turned out to discriminate craft better than a spoofed content_type alone.

The more interesting question comes after that. Since nobody wrote auth without instructions, how much instruction is enough? One sentence about trust boundaries in the prompt, or does every endpoint have to be spelled out separately? The same set of probes can measure that, so I will measure it.

The whole test is reproducible: the prompt is a dozen or so lines from the construction section, the probes are four curl commands, and the rubric fits on one page. If anyone runs it against other models, or the same ones at a different temperature, I would be glad to compare results.