[2409.19663] Identifying Knowledge Editing Types in Large Language Models


View a PDF of the paper titled Identifying Knowledge Editing Types in Large Language Models, by Xiaopeng Li and 7 other authors

View PDF
HTML (experimental)

Abstract:Knowledge editing has emerged as an efficient technology for updating the knowledge of large language models (LLMs), attracting increasing attention in recent years. However, there is a lack of effective measures to prevent the malicious misuse of this technology, which could lead to harmful edits in LLMs. These malicious modifications could cause LLMs to generate toxic content, misleading users into inappropriate actions. In front of this risk, we introduce a new task, Knowledge Editing Type Identification (KETI), aimed at identifying different types of edits in LLMs, thereby providing timely alerts to users when encountering illicit edits. As part of this task, we propose KETIBench, which includes five types of harmful edits covering most popular toxic types, as well as one benign factual edit. We develop four classical classification models and three BERT-based models as baseline identifiers for both open-source and closed-source LLMs. Our experimental results, across 42 trials involving two models and three knowledge editing methods, demonstrate that all seven baseline identifiers achieve decent identification performance, highlighting the feasibility of identifying malicious edits in LLMs. Additional analyses reveal that the performance of the identifiers is independent of the reliability of the knowledge editing methods and exhibits cross-domain generalization, enabling the identification of edits from unknown sources. All data and code are available in this https URL. Warning: This paper contains examples of toxic text.

Submission history

From: Xiaopeng Li [view email]
[v1]
Sun, 29 Sep 2024 11:29:57 UTC (934 KB)
[v2]
Tue, 1 Oct 2024 06:35:24 UTC (638 KB)

Source link

#Identifying #Knowledge #Editing #Types #Large #Language #Models

ai improve text ai text generator tools best ai text generators blues metronome brain bravida comfyui ndi data governance failures fivetran blind h&r block taxes generative ai insurance denials builds aipowered for fighting
Unlock the potential of cutting-edge AI solutions with our comprehensive offerings. As a leading provider in the AI landscape, we harness the power of artificial intelligence to revolutionize industries. From machine learning and data analytics to natural language processing and computer vision, our AI solutions are designed to enhance efficiency and drive innovation. Explore the limitless possibilities of AI-driven insights and automation that propel your business forward. With a commitment to staying at the forefront of the rapidly evolving AI market, we deliver tailored solutions that meet your specific needs. Join us on the forefront of technological advancement, and let AI redefine the way you operate and succeed in a competitive landscape. Embrace the future with AI excellence, where possibilities are limitless, and competition is surpassed.

Leave a Comment