AGB-DE: A Corpus for the Automated Legal Assessment of Clauses in German Consumer Contracts

Daniel Braun, Florian Matthes

Publikation: Beitrag in Buch/Bericht/KonferenzbandKonferenzbeitragBegutachtung

Abstract

Legal tasks and datasets are often used as benchmarks for the capabilities of language models. However, openly available annotated datasets are rare. In this paper, we introduce AGB-DE, a corpus of 3,764 clauses from German consumer contracts that have been annotated and legally assessed by legal experts. Together with the data, we present a first baseline for the task of detecting potentially void clauses, comparing the performance of an SVM baseline with three fine-tuned open language models and the performance of GPT-3.5. Our results show the challenging nature of the task, with no approach exceeding an F1-score of 0.54. While the fine-tuned models often performed better with regard to precision, GPT-3.5 outperformed the other approaches with regard to recall. An analysis of the errors indicates that one of the main challenges could be the correct interpretation of complex clauses, rather than the decision boundaries of what is permissible and what is not.

OriginalspracheEnglisch
TitelLong Papers
Redakteure/-innenLun-Wei Ku, Andre F. T. Martins, Vivek Srikumar
Herausgeber (Verlag)Association for Computational Linguistics (ACL)
Seiten10389-10405
Seitenumfang17
ISBN (elektronisch)9798891760943
PublikationsstatusVeröffentlicht - 2024
Veranstaltung62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024 - Bangkok, Thailand
Dauer: 11 Aug. 202416 Aug. 2024

Publikationsreihe

NameProceedings of the Annual Meeting of the Association for Computational Linguistics
Band1
ISSN (Print)0736-587X

Konferenz

Konferenz62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024
Land/GebietThailand
OrtBangkok
Zeitraum11/08/2416/08/24

Fingerprint

Untersuchen Sie die Forschungsthemen von „AGB-DE: A Corpus for the Automated Legal Assessment of Clauses in German Consumer Contracts“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren