Causal Scan for LLM Misbehavior Detection

View a PDF of the paper titled LLMScan: Causal Scan for LLM Misbehavior Detection, by Mengdi Zhang and three different authors

View PDF
HTML (experimental)

Summary:Regardless of the success of Massive Language Fashions (LLMs) throughout numerous fields, their potential to generate untruthful, biased and dangerous responses poses important dangers, significantly in essential purposes. This highlights the pressing want for systematic strategies to detect and stop such misbehavior. Whereas current approaches goal particular points akin to dangerous responses, this work introduces LLMScan, an revolutionary LLM monitoring approach primarily based on causality evaluation, providing a complete answer. LLMScan systematically displays the inside workings of an LLM by way of the lens of causal inference, working on the premise that the LLM’s `mind’ behaves otherwise when misbehaving. By analyzing the causal contributions of the LLM’s enter tokens and transformer layers, LLMScan successfully detects misbehavior. In depth experiments throughout numerous duties and fashions reveal clear distinctions within the causal distributions between regular conduct and misbehavior, enabling the event of correct, light-weight detectors for a wide range of misbehavior detection duties.