OpenAI has disrupted a coordinated campaign involving more than 15,000 user accounts that allegedly attempted to extract protected reasoning from its AI models, an activity the company described as adversarial model distillation.
The operation began in early July and escalated into high-volume extraction attempts later that month, prompting account bans, infrastructure controls, and expanded monitoring.
The campaign did not breach OpenAI’s infrastructure, encryption systems, databases, or stored user conversations. Instead, the operators reportedly manipulated model interactions to make hidden reasoning artifacts appear in user-visible outputs.
OpenAI Stops 15,000-User Operation Attempting
OpenAI said the activity violated its terms of service and was designed to collect reasoning data that could potentially be used to train, reproduce, or improve another AI model.
Protected reasoning refers to an internal record a model uses while processing a task. Unlike the final answer delivered to a user, this reasoning may contain intermediate analysis, structured task decomposition, and other information not intended for exposure.
OpenAI warned that extracting such material could enable third parties to replicate model capabilities while bypassing the safety controls applied to normal user-facing responses.
The company said it first observed suspicious activity on July 1 at relatively low volume. The operation intensified on July 24 and July 25, when OpenAI detected 16,000 requests using a relevant extraction pattern from more than 4,000 users.
Investigators later linked the activity to a broader network of over 15,000 accounts, which OpenAI said it had fully disrupted by July 28.
One technique involved copying encrypted reasoning material from one model conversation and submitting it to another conversation with instructions to decrypt and transcribe the hidden content.
OpenAI said independent researchers had separately disclosed related weaknesses involving cross-model interactions and conversation-compaction behavior. The company confirmed that the researchers’ attack paths were valid and said the disclosure helped accelerate mitigation work.
OpenAI attributed a core cluster of the activity to individuals associated with Moonshot AI, the developer of the Kimi AI model. However, the company said it could not determine whether every operator involved in the wider activity originated from a single actor or organization.
To contain the campaign, OpenAI banned or restricted fraudulent accounts, strengthened signup and infrastructure safeguards, and expanded detection for connected account networks.
It also closed a replay pathway that could allow someone already in possession of another user’s encrypted reasoning to recover its contents. The company added controls to detect and hold streamed model output that could expose hidden reasoning.
The company said it coordinated with third-party service providers where related activity passed through external platforms. It also shared findings with the Frontier Model Forum, industry partners, government information-sharing channels, and public-sector stakeholders.
OpenAI said adversarial distillation represents a growing AI security concern because extracted reasoning could transfer advanced capabilities without requiring equivalent investment in model development or safety engineering.
The risk grows as frontier models gain dual-use capabilities relevant to cybersecurity, scientific research, and other sensitive domains.
The company expects future extraction attempts to become more sophisticated and said ongoing work will focus on stronger technical protections, improved detection and enforcement, hardened tool-output defenses, broader classifier coverage, and propagation of protections across cloud and partner-hosted deployments.
Join 16,000+ SOC teams using ANY.RUN to streamline threat investigations and reduce manual effort. Explore for your team
