bibtext: @inproceedings{
  kumar2024certifying,
  title={Certifying {LLM} Safety against Adversarial Prompting},
  author={Aounon Kumar and Chirag Agarwal and Suraj Srinivas and Aaron Jiaxun Li and Soheil Feizi and Himabindu Lakkaraju},
  booktitle={First Conference on Language Modeling},
  year={2024},
  url={https://openreview.net/forum?id=9Ik05cycLq}
  }
abstract: Large language models (LLMs) are vulnerable to adversarial attacks, which add maliciously designed token sequences to bypass the model’s safety guardrails and cause it to produce harmful content. In this work, we introduce erase-and-check, the first framework to defend against adversarial prompts with certifiable safety guarantees. Given a prompt, our erase-and-check method erases tokens individually and inspects the resulting subsequences using a safety filter, declaring it harmful if any of the subsequences are detected as harmful. Our safety filters are implemented by leveraging Llama 2 and DistilBERT. We theoretically demonstrate that our method detects harmful prompts with accuracy at least as high as the safety filter. Additionally, we propose three efficient empirical defenses inspired by our erase-and-check (EC) method: i) RandEC, a randomized subsampling version of erase-and-check; ii) GreedyEC, which greedily erases tokens that maximize the softmax score of the harmful class; and iii) GradEC, which uses gradient information to optimize the tokens to erase. Extensive empirical evaluation with real-world datasets demonstrates the effectiveness of the proposed methods in defending against state-of-the-art adversarial prompting attacks.
title: Certifying LLM Safety against Adversarial Prompting
authors: Aounon Kumar Chirag Agarwal Suraj Srinivas Aaron Jiaxun Li Soheil Feizi Himabindu Lakkaraju