Paper accepted at ICAIL 2026: LLMs for Generating Decision Models from Legal Text

🎉 Full paper accepted at the 21st International Conference on Artificial Intelligence and Law (ICAIL 2026): “From Legal Text to Executable Decision Models: Evaluating Structured Representations for Legal Decision Model Generation”

**Update 10 Jun 2026: I presented this paper at ICAIL 2026, see my slides here.**

Get the preprint, here:

D. Graus, “From Legal Text to Executable Decision Models: Evaluating Structured Representations for Legal Decision Model Generation,” in Proceedings of the International Conference on Artificial Intelligence and Law (ICAIL), Singapore, 2026.
[Bibtex]

@inproceedings{graus2026legaltext,
author = {Graus, David},
title = {{From Legal Text to Executable Decision Models: Evaluating Structured Representations for Legal Decision Model Generation}},
booktitle = {{Proceedings of the International Conference on Artificial Intelligence and Law (ICAIL)}},
year = {2026},
month = jun,
address = {Singapore},
publisher = {ACM},
note = {To appear},
}

This paper studies LLMs and representation for generating executable decision models from legal text, using the Dutch Environment and Planning Portal (Digitaal Stelsel Omgevingswet (DSO)) as a case study, as it pairs complex hand-crafted decision models with legal articles. There’s 2 main findings!

1️⃣ We* find that providing input & output specifications (i.e., what variables a citizen must provide, and what the legal outcomes may be) improve LLM performance over raw legal text or text enriched with semantic role labels.

2️⃣ Interestingly, when evaluating generated models’ structural similarity to ground truth (do the models look similar?), and outcome similarity (do they yield the same legal outcomes?) we find complementarity: generated models can score well on structure while missing legal nuance, or vice versa. 🤏 More interestingly, we find that some models are structurally very dissimilar, and mostly much smaller/simpler, while yielding 100% identical legal outcomes! See attached pic for an example: the generated decision model yields 100% identical outcomes while completely ignoring 2 out of 4 input variables!

🎁 We release the full dataset of 95 production-grade decision models and their associated legal text, for reproducibility and extension of this work. It really is a pretty cool dataset!

Stay tuned for the preprint! See you in 🇸🇬 Singapore, where we’ll also host the 1st AI & Open Government (AIOG) Workshop!

* when I say “we”, I actually mean “I” but it sounds weird — this is a single author paper; but I have to thank Anne Schuth and Damiaan Reijnaers for feedback, help, and inspiration!

Related