Word-level Annotation of GDPR Transparency Compliance in Privacy Policies using Large Language Models
Journal:
arXiv
Published Date:
Mar 13, 2025
Abstract
Ensuring transparency of data practices related to personal information is a
fundamental requirement under the General Data Protection Regulation (GDPR),
particularly as mandated by Articles 13 and 14. However, assessing compliance
at scale remains a challenge due to the complexity and variability of privacy
policy language. Manual audits are resource-intensive and inconsistent, while
existing automated approaches lack the granularity needed to capture nuanced
transparency disclosures.
In this paper, we introduce a large language model (LLM)-based framework for
word-level GDPR transparency compliance annotation. Our approach comprises a
two-stage annotation pipeline that combines initial LLM-based annotation with a
self-correction mechanism for iterative refinement. This annotation pipeline
enables the systematic identification and fine-grained annotation of
transparency-related content in privacy policies, aligning with 21 GDPR-derived
transparency requirements. To enable large-scale analysis, we compile a dataset
of 703,791 English-language policies, from which we generate a sample of 200
manually annotated privacy policies.
To evaluate our approach, we introduce a two-tiered methodology assessing
both label- and span-level annotation performance. We conduct a comparative
analysis of eight high-profile LLMs, providing insights into their
effectiveness in identifying GDPR transparency disclosures. Our findings
contribute to advancing the automation of GDPR compliance assessments and
provide valuable resources for future research in privacy policy analysis.
