Ryoma Kondo, Riona Matsuoka, Takahiro Yoshida, Kazuyuki Yamasawa, Ryohei Hisano
August 24, 2025
PDF
The paper tackles legal reasoning by building a knowledge graph from 648 Japanese administrative court decisions that makes the hidden reasoning path machine-readable. The system uses large language models to extract the key components of legal reasoning: factual findings, legal provisions cited, and how the court applied those provisions to the facts, and connects them through a purpose-built legal ontology. The result is a structured graph where you can trace the logical steps from a fact to the legal norm it triggers to the outcome it produces. In retrieval tests, the system outperformed standard LLM baselines at finding the correct legal provisions given a set of facts, meaning the knowledge graph adds genuine precision beyond what a general-purpose AI can achieve alone.
Knowledge Graph (KG) is a database that stores information as a network of entities and relationships rather than rows and columns. In a legal context, entities might be facts, court decisions, legal provisions, parties, and the relationships between them capture how they connect (e.g., “Fact A triggers Provision B which leads to Outcome C”). Knowledge graphs make implicit relationships explicit and queryable.
Legal Reasoning Path is the structured logical chain a court follows from factual findings to a legal conclusion: facts → applicable legal norm → application of the norm to the facts → decision. In most court opinions this path is written as prose and must be inferred by a human reader. This paper’s core contribution is extracting and storing these paths as structured data.
Ontology is a formal specification of concepts and relationships within a domain — essentially a vocabulary with rules. A legal ontology defines what entities exist in legal reasoning (facts, norms, parties, outcomes) and how they can relate to each other. It constrains the knowledge graph so that extracted information follows a consistent structure across all cases.
Expert Annotation is created by having human domain experts (legal professionals) manually label examples to create a “gold standard” dataset for evaluating the system’s accuracy. The annotated examples serve as the benchmark. If the system’s extracted reasoning paths match what the experts identified, the system is considered accurate.