Be a Multitude to Itself: A Prompt Evolution Framework for Red Teaming
Citations
0
Open access
No
Source
arxiv
OpenAlex
Not enriched
arXiv
2502.16109
Abstract
Large Language Models (LLMs) have gained increasing attention for their remarkable capacity, alongside concerns about safety arising from their potential to produce harmful content. Red teaming aims to find prompts that could elicit harmful responses from LLMs, and is essential to discover and mitigate safety risks before real-world deployment. However, manual red teaming is both time-consuming and expensive, rendering it unscalable. In this paper, we propose RTPE, a scalable evolution framework to evolve red teaming prompts across both breadth and depth dimensions, facilitating the automatic generation of numerous high-quality and diverse red teaming prompts. Specifically, in-breadth evolving employs a novel enhanced in-context learning method to create a multitude of quality prompts, whereas in-depth evolving applies customized transformation operations to enhance both content and form of prompts, thereby increasing diversity. Extensive experiments demonstrate that RTPE surpasses existing representative automatic red teaming methods on both attack success rate and diversity. In addition, based on 4,800 red teaming prompts created by RTPE, we further provide a systematic analysis of 8 representative LLMs across 8 sensitive topics.
Collections
Add to collection
Paper intelligence
Analysis has not been completed yet.
No graph connections yet.
Sync citations or add papers to shared collections to build this network.
Knowledge graph
Citation network
Explore references, papers that cite this work and related papers in your Codex library.
References
0No references have been linked yet.
Cited by
0No saved paper is currently linked as citing this work.
Related papers
0Add papers to shared collections or enrich their topics to find related work.
Research workspace
Attach the paper PDF, extract its text, classify its contents and create semantic embeddings.
Paper resources
No PDF assets have been attached to this paper yet.