Portland firm shows hackers can alter AI answers through hardware tampering
Researchers at Eclypsium demonstrated that accessing specific data center components can cause AI models to give incorrect advice on health and national security.
By Dominic Foster ·
A Portland-based security firm has shown that hackers can manipulate the hardware running artificial intelligence models to force them to give dangerous or incorrect answers. The research, conducted by Eclypsium, demonstrates that tampering with the underlying infrastructure can subvert the output of major AI systems.
In one test, researchers remotely altered the data used by an AI model to process information. This change caused the model to drastically increase the estimated probability of a Chinese invasion of Taiwan. In a separate scenario, a user asked an AI for medical advice regarding chest pain. The model, running on tampered hardware, advised the user to stay home rather than seek hospital care.
Yuriy Bulygin, chief executive of Eclypsium, stated that trust in an AI model depends on trust in the infrastructure supporting it. The firm is staffed by security engineers with prior experience at major hardware companies. The study focused on the security of the multibillion-dollar hardware industry, including chips, memory, and servers, rather than the actions of the AI companies themselves.
Eclypsium demonstrated the attack on a computing platform made by Nvidia. The company noted that the attack scenario is not unique to Nvidia and could apply to other hardware and code. A Nvidia spokesperson said the firm reviewed the scenarios and did not identify anything the industry was not previously aware of. The spokesperson added that strong memory access protections can block the attack path.
The attack requires specific conditions to be met. An attacker generally needs network access to a data processing unit or a baseboard management controller. Certain security configurations must also be disabled. With access to these components, an attacker can read and modify code and data in the machine's memory. The researchers said the attack takes less than a minute and leaves the AI user unaware of the tampering.
Eclypsium executives briefed the National Security Agency and the Cybersecurity and Infrastructure Security Agency on the findings. Two independent hardware security experts reviewed the results for CNN. Joe FitzPatrick, a hardware security instructor, called the findings credible. Dominic Rizzo, a senior fellow at the Institute for Security and Technology, noted that the attack path is general to modern infrastructure.
Nvidia announced a new security platform last month designed to guard against hardware attacks. Eclypsium said the effectiveness of security controls depends on how they are configured and used. The firm is working with Nvidia to ensure these controls are effective. John Loucaides, a senior vice president at Eclypsium, urged the industry to remember the computers running the models, not just the model safety itself.
The research arrives as the race for AI supremacy between China and the US intensifies. Real-world cases of AI reliability issues are also increasing. This spring, a false AI-assisted report about a Chinese ship transporting nuclear components nearly led the US military to board the vessel. The new findings add to concerns about the integrity of AI systems in sensitive applications.
