Cybersecurity researchers have uncovered two malicious machine learning (ML) models hosted on Hugging Face, a popular hub for hosting and sharing machine learning models, leveraging a technique involving “broken” pickle files to evade detection systems.
According to a report by ReversingLabs, the pickle files extracted from the affected PyTorch archives revealed a malicious payload. The malicious payload was a typical platform-aware reverse shell that connects to a hard-coded IP address, the researchers explained.
This suggests that the models were intended to open backdoors into compromised systems, giving attackers the ability to remotely control the victim’s machine.
The method used to hide the payload is what has been dubbed ‘nullifAI’, a strategy designed to bypass existing safeguards aimed at detecting malicious models. The two Hugging Face repositories observed in this attack (glockr1/ballr7 and who-r-u0000/0000000000000000000000000000000000000) are believed to be more of a proof-of-concept (PoC) rather than part of an active supply chain attack.
Pickle is a Python-specific serialization format used to store and transmit Python objects. When a pickle file is loaded and deserialized, it can execute arbitrary code. In this case, the malicious models were stored in the PyTorch format, which is a compressed pickle file. Typically, PyTorch uses the ZIP format for compression, but in this instance, the models were compressed using the 7z format. This allowed the models to slip past security tools that might normally flag the files as suspicious.
After the malicious payload was executed, the object serialization in the pickle file would break, causing the object’s decompilation to fail.
“An interesting thing about this Pickle file is that the object serialization — the purpose of the Pickle file — breaks shortly after the malicious payload is executed, resulting in the failure of the object's decompilation,” the report noted.
However, the broken pickle file could still be partially deserialized, allowing the malicious code to run even if a security tool like Picklescan threw an error message. Picklescan, a tool used by Hugging Face to detect suspicious pickle files, had previously failed to flag these files as malicious.
After discovering this flaw, the open-source tool has been updated to prevent such evasion tactics in the future.