Objection, your honor!: an LLM-driven approach for generating Korean criminal case counterarguments

 

[en] Objection, your honor!: an LLM-driven approach for generating Korean criminal case counterarguments

 

Park, Sungmi and Choi, Ari and Park, Roseop

Artificial Intelligence and Law

 

This study explores the integration of advanced language models (LLMs) with legal argumentation processes, aiming to address a question posed by Walton in 2004: the potential of argumentation methods to identify the best arguments for supporting or refuting a claim. By leveraging the capabilities of LLMs, we demonstrate the practical application of argumentation generation methods in the legal domain, marking a shift from traditional retrieval-focused tasks to generation-focused endeavors. Our methodology diverges from existing literature by emphasizing the development of defeasible legal arguments, suitable for a hypothesis reduction process, rather than merely crafting effective rebuttals. Utilizing a Retrieval-Augmented Generation-based system, our approach simulates reliable legal argument generation by drawing from similar case data, thereby creating a network of interconnected arguments. These networks are formalized into argumentation schemes to compute argument acceptance and facilitate decision-making. To validate ALEX (Argumentation system for Legal Explanation), we conducted an extensive comparative study with state-of-the-art LLMs, including a fine-tuned model, evaluating the quality of generated arguments against original judicial decisions. Additionally, an expert survey evaluated the accuracy of case analysis and the practical alignment of generated arguments with real-world requirements, providing valuable insights into the system’s performance. Our research also includes a detailed case analysis, tracing the argumentation generation process step by step, and providing recommendations for future improvements. This study showcases ALEX’s capacity to improve legal analysis through automated argumentation, enhanced argument quality, and the structuring of arguments into formal frameworks, with demonstrated benefits in real-world applications. By integrating traditional legal practices with advanced LLM capabilities, ALEX refines legal reasoning and establishes a solid foundation for the evolution of automated legal argumentation systems.