Acoustic Interference: A New Paradigm Weaponizing Acoustic Latent Semantic for Universal Jailbreak against LALMs (ICML'26)

The core philosophy of our method is to shift the attack vector from optimizing malicious audio to interfering with safety alignment.

Unlike previous works that rely on discrete, pre-defined proxies of audio features, such as simple “happy” or “angry” tags for emotion, which may map poorly to the high-dimensional acoustic space, the proposed ALS is constructed by mining a native manifold of neural AGMs. This is expected to fit more the native latent space of LALMs, thus more effectively serving as the weapon to reveal their vulnerability.

Below is the exploration process for the vulnerability of LALMs to the proposed Acoustic Interference. The results show a bi-directional interference effect. The introduction of ALS suppresses the success of previously strong text attacks but amplifies that of originally relatively weak ones, indicating that even natural ALS can cause a drift in the safety alignment path of LALM inference.

Here are the attack results of AIA upon seven open-source and three proprietary LALMs on the JBB and WildJailbreak datasets. We report two ASR metrics (as detailed in Section 3) and the average query times. For each AIA entry, we also provide text-only jailbreak results for comparison, with the absolute ASR gain over them reported in parentheses. This demonstrates that the proposed AIA consistently amplifies the scores across all evaluated models, thus shaping a new general threat paradigm against LALMs: When text-only jailbreak reaches a bottleneck, the introduction of acoustic interference would induce a significant inference path drift, successfully bypassing the LALM safety alignment.

Below is the comparison of AIA on ASR and query time with the existing seven instance-specific and two universal audio jailbreak methods across JBB, AdvBench, HarmBench datasets and 11 popular LALMs. Among our main related works, the universal LALM jailbreaks, the best results on each LALM are highlighted in bold font, while those among instance-specific methods are underlined. The scores colored in gray are from JALMBench with a looser evaluation strategy (thus should be only explained in the manner detailed in Section 3.1). The query time of AJailBench is marked as “10+B” as it states the need for 10 startup queries plus several Bayesian-optimization queries, without specifying the exact number of the latter (empirically, such optimization can be expensive). Overall, the proposed AIA not only significantly builds new SOTA in universal LALM jailbreak, but even also effectively outperforms existing instance-specific methods in most cases. At the same time, it maintains a middle query time, ranking second among the three universal methods and fifth among all 10 methods.

We also investigate the specific ALS patterns that render LALMs more vulnerable, which is expected to provide prior knowledge to facilitate relevant studies in the LALM jailbreak and safety alignment community. This is based on the distribution divergence of acoustic features across the jailbreak outcomes. Specifically, we partition the ALS arsenal into the Top 25% (highest ASR, red) and Bottom 25% (lowest ASR, blue) successful ALS-synthesized interference audio. Most indexes demonstrate a significant impact on the jailbreak result, while intuitively, the larger the gray fields, the greater the impact.

To better facilitate the reproduction of our study and potential future works, in addition to the open-source code, we also provide the "Top 30" universal interference audio adopted in our main experiments. They can be directly appended to any malicious text prompts to perform robust jailbreak attacks against LALMs. Welcome to have a try!

This study involves potentially offensive and harmful content. Please only engage with it in accordance with your own personal risk tolerance. The open-source code and data are intended for research purposes only, especially for making models safer in the future.