In a concerning development for the artificial intelligence (AI) and machine learning (ML) community, security researchers have uncovered malicious ML models on Hugging Face, a widely used platform for sharing and collaborating on AI projects.
The discovery, dubbed “nullifAI,” highlights a novel attack method leveraging vulnerabilities in Python’s Pickle serialization format to evade detection and execute harmful code on unsuspecting systems.
Exploiting Pickle Files
Pickle, a popular Python module for serializing and deserializing data, is often used to store ML models.
However, its flexibility comes at a cost: it allows the execution of arbitrary Python code during deserialization.
This makes it an attractive target for cybercriminals seeking to embed malicious payloads into shared models.
Despite warnings in Hugging Face’s documentation about the risks of using Pickle files, they remain prevalent due to their simplicity and compatibility.
Recent investigations by ReversingLabs revealed two compromised models hosted on Hugging Face that bypassed the platform’s security measures.
These models contained malicious code designed to establish reverse shells, granting attackers remote access to compromised systems.
The payloads were embedded in corrupted Pickle files, which evaded detection by exploiting limitations in Hugging Face’s Picklescan tool a mechanism that relies on blacklists of known malicious functions but struggles with incomplete or broken files.
Broader Implications for AI Security
The issue extends beyond isolated incidents. JFrog researchers identified over 100 malicious models on Hugging Face capable of executing harmful code, posing risks such as data breaches, system corruption, and espionage.
Most of these attacks target PyTorch-based models due to their reliance on Pickle serialization, though TensorFlow Keras models are also vulnerable.
These findings underscore the broader vulnerabilities in open-source AI ecosystems.
Platforms like Hugging Face rely on community collaboration, making it challenging to ensure the integrity of shared resources.
While Hugging Face has introduced measures such as malware scanning and a safer serialization format called Safetensors, these steps are not foolproof.
Malicious actors continue to exploit gaps in security protocols, emphasizing the need for robust defenses.
To mitigate risks, developers are advised to:
- Avoid using untrusted Pickle files or models from unknown sources.
- Transition to safer serialization formats like Safetensors where possible.
- Implement additional security checks within their machine learning operations (MLOps) pipelines.
- Monitor for suspicious activity and regularly update security tools.
Hugging Face has responded swiftly by removing identified malicious models and updating its Picklescan tool to detect broken files more effectively.
However, experts warn that the fundamental insecurity of Pickle serialization remains a challenge that requires ongoing vigilance.
As AI continues to integrate into critical systems across industries, the stakes for securing its supply chain have never been higher.
This incident serves as a stark reminder of the evolving threats facing the AI community and the importance of prioritizing security alongside innovation.
%20(1).webp?w=1200&resize=1200,0&ssl=1 "Data Breach at Legends International Exposes Customer Information")
%20(1).webp?w=1200&resize=1200,0&ssl=1 "Critical AnythingLLM Vulnerability Enables Remote Code Execution")
