A Study of Automated Drafting of Legal Complaints Using Large Language Models: -Focusing on Loan and Merchandise Fraud-  

 

[ko]  A Study of Automated Drafting of Legal Complaints Using Large Language Models: -Focusing on Loan and Merchandise Fraud-  

 

Park, Jeewon and Cho, Eun-Byul and Park, RoSeop

Journal of Police & Law 2024, vol.22, no.3, p. 179-214  

With the recent amendment of Korea's Criminal Procedure Act and the abolition of complaint rejections, law enforcement agencies face an increasing investigative burden. This study explores the use of Large Language Models (LLMs), particularly GPT-4, to automate the drafting of criminal complaints, focusing on the generation of legally structured crime facts for loan fraud and merchandise fraud cases.

To construct a training dataset, legally significant information was extracted from over 3,000 anonymized court rulings using expert-developed guidelines and prompt engineering techniques, including Chain-of-Thought (CoT) and few-shot prompting. The extracted data were used to fine-tune two Korean-specialized LLMs (Fact_Gen_Mistral and Fact_Gen_Llama3), which demonstrated superior performance compared to baseline models. The generated crime facts showed high fidelity to original legal documents, with BLEU scores of up to 0.86 and cosine similarity reaching 0.94.

Error analysis revealed issues such as partial extractions and label misclassifications, mainly due to the complex structure of legal text and subtle differences in legal terminology. However, hallucination errors were effectively eliminated through strict prompt constraints and legal schema guidance. The findings validate the feasibility of LLM-based automation for generating structured, legally sound complaint documents. This approach holds significant potential to improve investigative efficiency and accuracy in the criminal justice system.