Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models

Hao Cheng1✶, Erjia Xiao1✶, Chengyuan Yu3, Yao Zhao4, Jiahang Cao1, Qiang Zhang1,
Jiaxu Wang1, Mengshu Sun6, Kaidi Xu5, Jindong Gu2, Renjing Xu1†
1The Hong Kong University of Science and Technology (Guangzhou)
2University of Oxford
3Hohai University 4Hunan University 4Drexel University 4Beijing University of Technology

✶ indicates equal contribution

arXiv Video

Abstract

Recently, driven by advancements in Multimodal Large Language Models (MLLMs), Vision Language Action Models (VLAMs) are being proposed to achieve better performance in open-vocabulary scenarios for robotic manipulation tasks. Since manipulation tasks involve direct interaction with the physical world, ensuring robustness and safety during the execution of this task is always a very critical issue. In this paper, by synthesizing current safety research on MLLMs and the specific application scenarios of the manipulation task in the physical world, we comprehensively evaluate VLAMs in the face of potential physical threats. Specifically, we propose the Physical Vulnerability Evaluating Pipeline (PVEP) that can incorporate as many visual modal physical threats as possible for evaluating the physical robustness of VLAMs. The physical threats in PVEP specifically include Out-of-Distribution, Typography-based Visual Prompt, and Adversarial Patch Attacks. By comparing the performance fluctuations of VLAMs before and after being attacked, we provide generalizable Analyses of how VLAMs respond to different physical security threats.

Framework



Overview of Framework. The above figure illustrates the overall framework for evaluating physical security threats to VLAMs using the Physical Vulnerability Evaluating Pipeline (PVEP).


Experiment Results


LLaRA Results


LLaRA Results: Under 3 physical attack categories: (left) Time steps (with a maximum limit of 8) of LLaRA on 14 VIMA tasks that are listed in TABLE I. (right) Failure rates of the OOD attacks with other levels that are not listed in TABLE I


OpenVLA Results


OpenVLA Results: Under 3 physical attack categories: (left) Time steps (with a maximum limit of 300) of OpenVLA on 6 SimplerEnv tasks that are listed in TABLE II. (right) Failure rates of the OOD attacks with other levels that are not listed in TABLE II.



Demo

LLaRA Manipulation
OpenVLA Manipulation
These videos are the attacked demos of LLaRA and OpenVLA. The attacking types are blurring, gaussian noise, brighter, darker, visual prompt, and adversarial patch.

BibTeX

@article{cheng2024manipulation,
  title={Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models},
  author={Cheng, Hao and Xiao, Erjia and Yu, Chengyuan and Yao, Zhao and Cao, Jiahang and Zhang, Qiang and Wang, Jiaxu and Sun, Mengshu and Xu, Kaidi and Gu, Jindong and others},
  journal={arXiv preprint arXiv:2409.13174},
  year={2024}
}