Inferring Safe Actions for LLM-Based Agents Through Preemptive Evaluation and Human Feedback


View a PDF of the paper titled InferAct: Inferring Safe Actions for LLM-Based Agents Through Preemptive Evaluation and Human Feedback, by Haishuo Fang and 2 other authors

View PDF
HTML (experimental)

Abstract:A crucial requirement for deploying LLM-based agents in real-life applications is the robustness against risky or even irreversible mistakes. However, the existing research lacks a focus on preemptive evaluation of reasoning trajectories performed by LLM agents, leading to a gap in ensuring safe and reliable operations. To explore better solutions, this paper introduces InferAct, a novel approach that leverages the belief reasoning ability of LLMs, grounded in Theory-of-Mind, to proactively detect potential errors before risky actions are executed (e.g., `buy-now’ in automatic online trading or web shopping). InferAct acts as a human proxy, detecting unsafe actions and alerting users for intervention, which helps prevent irreversible risks in time and enhances the actor agent’s decision-making process. Experiments on three widely-used tasks demonstrate the effectiveness of InferAct, presenting a novel solution for safely developing LLM agents in environments involving critical decision-making.

Submission history

From: Haishuo Fang [view email]
[v1]
Tue, 16 Jul 2024 15:24:44 UTC (8,557 KB)
[v2]
Thu, 17 Oct 2024 11:26:10 UTC (7,856 KB)

Source link

#Inferring #Safe #Actions #LLMBased #Agents #Preemptive #Evaluation #Human #Feedback


Unlock the potential of cutting-edge AI solutions with our comprehensive offerings. As a leading provider in the AI landscape, we harness the power of artificial intelligence to revolutionize industries. From machine learning and data analytics to natural language processing and computer vision, our AI solutions are designed to enhance efficiency and drive innovation. Explore the limitless possibilities of AI-driven insights and automation that propel your business forward. With a commitment to staying at the forefront of the rapidly evolving AI market, we deliver tailored solutions that meet your specific needs. Join us on the forefront of technological advancement, and let AI redefine the way you operate and succeed in a competitive landscape. Embrace the future with AI excellence, where possibilities are limitless, and competition is surpassed.