News

From Eugenio Vaccari, Senior Lecturer in Law, Royal Holloway University of London

Legal advice generated by Large Language Models (LLMs) may prove unsound, erroneous, or even absurd. According to a recent study, gpt-4 and other similar public models: “hallucinate at least 58% of the time, struggle to predict their own hallucinations, and often uncritically accept users’ incorrect legal assumptions.” 

To address these problems, as well as an unmet request for professional and accurate legal advice from both laypersons and professionals with limited experience in a technical and narrow area of law like insolvency law, our research group developed and tested the performance of a retrieval augmented generation (RAG) system for answering legal queries related to corporate insolvency in England and Wales. 

The Insolvency Bot we created (https://fastdatascience.com/insolvency/) relies on open-source legal information and HMRC forms to provide sound responses to a user’s query focusing on insolvency matters regulated by English law. According to a tailor-made mark scheme, we show that our Insolvency Bot consistently outperforms LLMs queried without our RAG setup, and we show that newer versions of LLMs consistently outperform older ones when queried with the Insolvency Bot’s RAG enhancements, and its data sources. 

Our legal chatbot demonstrates potential to ensure “access to justice” to affected business owners by combining a generative AI system with a trusted knowledge base. However, the system is not perfect. To improve its functionality, we are constantly updating the sources on which the system relies to provide advice on English corporate insolvency law issues. We are also exploring the possibility of extending the scope of the chatbot from corporate to personal bankruptcy matters, as these two areas are strictly linked whenever non-incorporated businesses face financial distress.

Nowadays, however, businesses, irrespective of their size, operate in an interconnected world. Business transactions between the UK and Europe, as well as within the Internal Market, are becoming increasingly common for small- and micro-enterprises. These entities, however, are ill-equipped to understand the legal complexities of cross-border insolvency disputes, should one of their clients and/or suppliers enter into a restructuring or insolvency procedure.

To address these issues, thanks to the support of INSOL Europe, we have been able to put together a working group of 14 experts across key jurisdictions in Europe to create a European Insolvency Chatbot. With this project, we aim to expand the Insolvency Bot expertise to cover cross-jurisdictional and insolvency-related queries that affect companies operating in selected European countries (France, Germany, Ireland, Italy, Netherlands and Spain). The aim of the project is to expand the knowledge of our existing chatbot to cross-border European disputes by March 2026, and to present the results of our work at the 2026 INSOL Europe Academic Colloquium, as well as in an academic paper.

---