What Happened
Security researcher HD Moore, chief research officer at the security firm Dragos, sent nearly two million AI prompts containing unique, trackable secrets to AI providers, then watched where those secrets showed up. According to Moore’s write-up, reported by Ars Technica, the results revealed security breaches, prompt theft, and providers ignoring customer privacy settings. The secrets included Amazon Web Services access keys tied to empty accounts, passwords, unique web addresses, and references to private code repositories, each placed in exactly one prompt sent to one provider.
Because every secret was used only once, any later use of it pointed straight back to the link in the chain that leaked it. Researchers call these “canaries,” after the birds once taken into coal mines.
How AI Requests Actually Travel
Many AI apps don’t talk directly to a big provider like OpenAI or Anthropic. They go through brokers, sometimes called AI routers or AI gateways, such as OpenRouter or Vercel. The router reads your prompt, then sends it to whichever model or hosting provider fits best, often based on price or task.
Here’s the catch Moore highlights: prompts are encrypted while traveling, but they are decrypted at each stop. The router reads the plain text. The request may then go to a pool of models running at different hosting companies, and sometimes from one router to another. Each of those parties may log the prompt and keep it as long as its policies allow.
What the Researcher Found
Moore reported three main problems:
- At least three AI providers were compromised by attackers who copied the prompts they received. At one provider, attackers reused “a couple hundred” of the bait AWS keys, and every request sent during certain windows in late October was stolen. In two of the breaches, some stolen requests were marked Zero Data Retention (ZDR). As Moore put it, a privacy flag doesn’t help when the attacker is already inside.
- Some resellers logged prompts themselves. They offered low prices to attract traffic, turned on logging, and the credentials later got used.
- Some providers ignored privacy settings. Moore says a Chinese AI lab’s systems fetched secret URLs from more than half of the prompts sent to its models between July and October, including roughly two-thirds of requests marked “no data collection” or ZDR. Several US hosting providers resell that lab’s models, and prompts sent through them triggered the same scraper. One US hosting provider crawled a secret URL more than two months after it appeared in a single request.
The stolen bait keys were used “tens of thousands” of times, with attackers probing for more credentials and trying to keep access. Note that the reporting does not name the breached providers, and it does not say the routers mentioned above were among them.
Why This Matters for Your Business
If your team pastes customer details, contracts, passwords, or internal links into AI tools, that information may pass through more hands than you realize. A cheap or convenient AI service might be routing your data through resellers you have never heard of. And a checkbox saying “don’t keep my data” is only as good as every company in the chain honoring it.
This connects to a broader theme we have covered: your business data has real value to others, which is why AI companies are eager to buy work files and emails. The canary study shows that value can also attract theft.
Practical Takeaways
1. Never put secrets in prompts
Passwords, API keys, access tokens, and private links should not go into any AI tool. That was the exact bait in this experiment, and it got used within hours in some cases. Make this a written rule for staff.
2. Know who actually processes your data
Moore’s advice: the most protective option is hosting models on your own servers, though he notes that is expensive. The next best option is working directly with a first-tier provider, giving you “one contract and a privacy policy to read.” He cautions that even this does not guarantee nothing leaks, but it reduces the risk. If you build with a router, he suggests choosing upstream providers yourself rather than accepting default pools.
3. Ask your vendors the routing question
If a software vendor added AI features to a tool you use, ask which model providers handle your data, whether requests pass through resellers, and where the data is processed. A US vendor may still send prompts to models that report back elsewhere, as the study found.
The PAD Take
This week, list every AI tool your team uses and ask each vendor in writing which providers actually process your prompts. Ban secrets and credentials in prompts outright, and favor direct contracts with major providers over the cheapest routed option for anything sensitive. If you want help mapping where your data goes, our AI checkup for reviewing your current tools is a practical place to start.

