Back to Dashboard
4.1.1 Privacy-Preserving Third-Party Access to Datasets
Original Problem in the Paper
The paper frames external training-dataset access as important for external data audits that can identify “harmful, personal, or inappropriate data” and study how training data shapes model behavior. It also says unrestricted access can reproduce copyrighted data, leak sensitive intellectual property, or disclose illegal collection. The open problem is how to provide “sufficiently deep access” for auditing while protecting data-subject privacy, and how to reconcile auditing with privacy-preserving ML settings where “the developer themselves lacks such visibility,” such as federated or encrypted training data.
July 2026 Update & Trajectory
Narrow operationalization exists: in specific domains, secure data environments, remotely attested enclave systems, and query-based or federated AI-audit tooling can let approved researchers or auditors run approved computations over sensitive or proprietary assets without receiving unconstrained raw-data access. These mechanisms are not yet the same thing as standardized independent access to frontier-model training corpora. The sources reviewed here do not establish a general legal or technical mechanism that gives independent auditors deep, standardized access to major frontier-model training corpora while resolving copyright, trade-secret, privacy, federated/encrypted-provenance, and query-leakage constraints.
Verified 2026 status: OpenSAFELY is live; OpenMined describes active open-source privacy-preserving AI-audit tooling and partner work; and NAIRR Secure is an active pilot/demonstration effort for sensitive-data AI research. I did not verify a 2026 legal or technical standard that compels or normalizes AI training-dataset audit access across major labs.
Deployed / Operationalized
- Healthcare secure research environments: OpenSAFELY is a live platform for analysing large, sensitive NHS-related datasets safely and securely. It reports 200+ projects, 100+ published outputs, and 30+ organisations, and its explanatory page says researchers bring code to data, receive aggregated tables or graphs, and never get unconstrained raw-data access.
- OpenMined says its open-source AI-auditor software enables external researchers to evaluate proprietary AI systems and user interactions by answering specific questions without directly observing or acquiring the system or revealing information beyond those approved answers. Its PySyft materials describe privacy-preserving queries over production algorithm information on secure servers, and an OpenMined AI-audit tutorial proposes approved query workflows over AI-system assets such as user logs, training data, models, compute allocation, APIs, and metadata.
- Project Oak provides building blocks for externally verifiable enclave applications, remote attestation, encrypted and attested channels, transparent release/provenance, and sealed-computing patterns that could support privacy-preserving audit computation.
- NAIRR Secure is a pilot for sensitive-data AI research that explores secure environments, privacy/security-preserving infrastructure demonstrations, and controlled-access AI-ready data use cases, with limited current sensitive-data provider capacity.
New Tractable Vectors
- Design auditor query languages with privacy budgets, output checking, and statistical-power targets for harmful-content, copyrighted-data, personal-data, and inappropriate-data audits.
- Prototype combinations of TEEs, differential privacy, secure multiparty computation, and transparent release/provenance logs so auditors can verify computation identity and some data-use constraints.
- Develop methods to audit federated or encrypted training data by producing verifiable aggregate evidence about data composition, licenses, and sensitive attributes without requiring auditors to decrypt all records.
- Develop dataset cards and provenance summaries whose claims can be checked by privacy-preserving spot queries rather than accepted on blind trust.
Key Open Questions
- Prevent adaptive-query reconstruction or membership inference by auditors while preserving enough audit depth to make the access meaningful.
- Resolve who can approve access, and who bears liability, when datasets may contain copyrighted, personal, illegal, or third-party-licensed material.
- Make confidential-computing audit stacks robust against side channels, TEE vendor trust failures, and proprietary host compromise.
- Define minimum disclosure and audit rights for frontier training datasets when data is distributed, synthetic, encrypted, or no longer retained.
Evidence & Primary Sources
- Reuel et al., *Open Problems in Technical AI Governance*, section 4.1.1, directly frames external dataset access as crucial for audits of harmful, personal, or inappropriate data; warns that unrestricted access can reproduce copyrighted data, leak sensitive IP, or disclose illegal collection; and names the unresolved problem of sufficiently deep access with data-subject privacy, including federated/encrypted settings where developers may lack visibility: https://arxiv.org/abs/2407.14981
- OpenSAFELY is a live platform for analysing large, sensitive datasets safely and securely. Its homepage reports 200+ projects, 100+ published outputs, and 30+ organisations; the page read here listed a latest blog dated 1 July 2026 and footer ©2026: https://www.opensafely.org/
- OpenSAFELY’s “in brief” page says raw data stays where it is stored, OpenSAFELY takes research code to the data and returns aggregated tables/graphs, researchers see insights rather than unconstrained raw data, and NHS England controls applications for the NHS England OpenSAFELY service: https://www.opensafely.org/os-in-brief/
- OpenMined’s AI-auditor page says it builds open-source software enabling external researchers to evaluate proprietary AI systems without directly observing or acquiring the systems, and to answer specific questions about closed-source systems and interactions with users without revealing information outside those answers; the page metadata shows modification in 2026: https://openmined.org/for-ai-auditors/
- OpenMined’s PySyft/DSIT post says PySyft can enable external researchers to study proprietary algorithmic systems while protecting proprietary information and private user data; model owners can load production AI algorithm information onto a secure server; researchers can query in privacy-preserving ways; and answers return without access to underlying sensitive information. It also says OpenMined deployed PySyft at Dailymotion and LinkedIn through CCIAO demonstrations: https://openmined.org/blog/pysyft-featured-in-uk-dsit-portfolio-of-ai-assurance-techniques/
- OpenMined’s AI-audit tutorial proposes a query-based system in which an external auditor asks an approved question and downloads only the answer, while AI owners and third parties do not learn or reveal beyond the approved information. It also discusses user logs, training data, models, compute allocation, APIs, metadata, secure enclaves, and secure multiparty computation as AI-audit infrastructure concepts; because it is a 2023 tutorial/proposal, it supports technical positioning rather than broad 2026 standardization: https://openmined.org/blog/ai-audit-part-1/
- Project Oak describes a software platform for externally verifiable claims about distributed-system behavior, including Enclave Applications isolated from the host and remotely attestable; TEEs such as AMD SEV-SNP and Intel TDX; encrypted and attested channels; split architecture; transparent release/provenance concepts; and sealed computing for privacy-preserving applications: https://github.com/project-oak/oak
- NAIRR Secure says the pilot enables AI research involving sensitive data requiring secure computing environments; is co-led by DOE and NIH; explores requirements and infrastructure design patterns for future secure resources, privacy/security-preserving data combination, and demonstration projects; and notes that only a limited number of resource providers can currently accommodate sensitive-data requests such as HIPAA-compliant research needs: https://nairrpilot.org/nairr-secure
- NSF’s NAIRR page says NAIRR Secure is led by NIH and DOE and strengthens secure, privacy-preserving AI research by exploring privacy/security-preserving infrastructure and secure environments for high-quality AI-ready controlled-access data. The page also includes a 2026 two-year progress update and broader NAIRR scale information, but it does not establish a legal or technical standard for frontier-lab training-dataset audit access: https://www.nsf.gov/focus-areas/ai/nairr