Researchers found that large language models can invent believable but fake web domains, and attackers can register those domains before defenders notice.
In this case, the threat actor used an AI coding assistant to build a phishing kit called Montana Empire and aimed it at a domain that had already been identified as a high-risk hallucination target 23 days earlier.
The result is a closed loop where AI helps both sides of the attack, one model suggests the fake domain, and another AI tool helps build the phishing infrastructure around it.
The kit was not a simple fake login page. It included a full brand clone, a PHP backend, a real-time storefront scraper, credential capture logic, and a Telegram-based command channel for the operator.
Researchers also found support for one-time password relay, victim handling, and payment credential theft, which shows the kit was built for active fraud rather than basic credential harvesting.
AI-Hallucinated Domain Theft
This campaign highlights a new phishing risk tied to AI hallucinations, often called phantom squatting. Instead of relying only on email lures or typo-squatted domains, attackers can exploit URLs that an AI system itself confidently invents and then get users to trust them.
That makes the fake domain feel more credible because it appears to come from an assistant or automation workflow the victim already trusts.
Paloaltonetworks said, the broader research behind the case is also significant.
The analysts tested 913 global brands, generated 2.1 million unique URLs, and found 13,229 confirmed malicious URLs plus about 250,000 unregistered hallucinated domains that could be registered later by criminals.
For defenders, the key lesson is simple, AI-generated links should never be trusted without verification, especially in workflows that involve credentials, payments, APIs, or automated agents.
Montana Empire shows how AI hallucinations can become real attack infrastructure, turning a fake domain into a working credential theft operation.
The campaign proves that defenders must monitor AI-generated domains early, because attackers can register them before reputation-based security tools react.
Follow us on Google News , LinkedIn and X to Get More Instant Updates. Set Cyberpress as a Preferred Source in Google.
