Artificial Reason and Artificial Intelligence: the Legal Reasoning Capabilities of GPT-4

Abstract

Despite the widespread adoption of generative transformer large language models and the interest of the global legal community, discussions about the models in philosophy of law mainly have been focusing on what LLMs cannot do. In making the first steps towards a philosophical analysis of the capabilities of AI models in the field of law, we follow the basic idea of Turing’s „imitation game“. Proceeding from the frequently raised characterization of legal reasoning as „artificial“, the paper identifies the undisputed minimum core of the „artificiality“ thesis and asks to what extent it can be imitated by artificial intelligence. To answer this question, we test the legal reasoning capabilities of ChatGPT, the most advanced, up-to-date LLM version of artificial intelligence. The conclusion is that in all relevant types of activities usually associated with legal reasoning – fact-finding, interpretation, qualification, and decision making – ChatGPT can generate outcomes as if it reasons legally.