Red Teaming for Generative AI, Report on a Copyright-Focused Exercise Completed in an Academic Medical Center
Journal:
arXiv
Published Date:
Jun 26, 2025
Abstract
Background: Generative artificial intelligence (AI) deployment in academic
medical settings raises copyright compliance concerns. Dana-Farber Cancer
Institute implemented GPT4DFCI, an internal generative AI tool utilizing OpenAI
models, that is approved for enterprise use in research and operations. Given
(1) the exceptionally broad adoption of the tool in our organization, (2) our
research mission, and (3) the shared responsibility model required to benefit
from Customer Copyright Commitment in Azure OpenAI Service products, we deemed
rigorous copyright compliance testing necessary.
Case Description: We conducted a structured red teaming exercise in Nov.
2024, with 42 participants from academic, industry, and government
institutions. Four teams attempted to extract copyrighted content from GPT4DFCI
across four domains: literary works, news articles, scientific publications,
and access-restricted clinical notes. Teams successfully extracted verbatim
book dedications and near-exact passages through various strategies. News
article extraction failed despite jailbreak attempts. Scientific article
reproduction yielded only high-level summaries. Clinical note testing revealed
appropriate privacy safeguards.
Discussion: The successful extraction of literary content indicates potential
copyrighted material presence in training data, necessitating inference-time
filtering. Differential success rates across content types suggest varying
protective mechanisms. The event led to implementation of a copyright-specific
meta-prompt in GPT4DFCI; this mitigation has been in production since Jan.
2025.
Conclusion: Systematic red teaming revealed specific vulnerabilities in
generative AI copyright compliance, leading to concrete mitigation strategies.
Academic medical institutions deploying generative AI should implement
continuous testing protocols to ensure legal and ethical compliance.
