4.3.1 Facilitation of Third-Party Access to Models
Original Problem in the Paper
External research and evaluation of AI systems require access to the underlying models, but many systems are not openly released and current APIs often lack the depth or flexibility needed for research and auditing. The source paper says black-box-only evaluations can “produce misleading results” and offer only “limited insights” into failures. Its open problems include mapping which research and auditing methods are possible across a continuum from black-box to white-box access; assessing how different access forms affect misuse, model-theft, privacy, and security risks; reconciling research/audit access with commercial and safety concerns; exploring PET/TEE/MPC-style near-white-box auditing; and preserving stable access to hosted model versions for reproducible research.
July 2026 Update & Trajectory
Third-party model access is more institutionally visible than in the original paper’s baseline: UK AISI evaluations, voluntary Seoul commitments, and frontier-lab safety frameworks create channels for pre/post-deployment testing or confidential sharing with trusted actors. The cited evidence supports institutional channels, not broad independent access or a quantified prevalence increase. Technical secure-access mechanisms exist in research/prototype form but are not broadly standardized; in the cited examples, frontier-lab access is mainly voluntary, policy-based, or partnership-based rather than a universal mandate. The core tradeoffs remain unresolved: how much grey/white-box access is needed for strong audits; how much extra model-theft or misuse risk is created by logits, fine-tuning APIs, limited internal access, or sandboxed/enclave-mediated weights; how to protect evaluator test suites and provider security; and how to preserve version-stable hosted-model access for reproducibility. The sources reviewed here do not establish a universal mandated near-white-box access regime as of July 2026.
Deployed / Operationalized
- UK AISI conducts government-led evaluations of advanced AI systems and says it assesses potential risks of new models before and after deployment. Its methods include automated capability assessments, red-teaming, human uplift evaluations, agent evaluations, and safeguard evaluations; it says methodology details are kept confidential, it cannot evaluate all released models, and it is not a regulator.
- The voluntary Seoul Frontier AI Safety Commitments commit signatories to internal and external red-teaming, to considering independent third-party or home-government evaluations where appropriate, to publishing safety frameworks, and to sharing more detailed nonpublic information with trusted actors where public disclosure would be unsafe or commercially sensitive.
- Anthropic’s Responsible Scaling Policy evolved through 2026; v3.2 authorized the LTBT to request external review of Risk Reports and approve external reviewer selection, and v3.3 revised the novel chemical/biological weapons threshold, showing an internal governance pathway for trusted external review.
- OpenAI’s Preparedness Framework v2 states that OpenAI uses scalable automated evaluations, expert-led deep dives, Capability Reports, Safeguards Reports, Safety Advisory Group review, and publication of Preparedness findings with frontier model releases.
- OpenMined/Syft and secure-enclave approaches offer early prototypes for privacy-preserving privileged evaluations: OpenMined’s auditor page claims external researchers can evaluate proprietary systems without directly observing or acquiring copies, and a 2024 OpenMined/AISI/Anthropic enclave pilot tested a workflow with proxy assets while leaving scale, side-channel, and architecture-protection issues open.
New Tractable Vectors
- Define a working access-tier taxonomy—e.g., black-box/API, tool-enabled API, logit access, sandboxed fine-tuning, limited internals/activation access, enclave-mediated weight access, and reproducible snapshots—and validate which audit methods and risks attach to each tier.
- Develop and validate TEE/MPC-based near-white-box evaluations in which auditors can run agreed tests without copying model weights and with controlled disclosure of evaluator test suites to providers.
- Create versioned model escrow or deprecation protocols for reproducibility of hosted models, including documented access to deprecated snapshots where safety, legal, and commercial constraints permit.
- Quantify marginal model-theft and misuse risk from black-box distillation, logits, fine-tuning APIs, limited internal/activation access, and sandboxed or enclave-mediated weight access; add separate evidence before including embeddings.
Key Open Questions
- Trusted-evaluator access is not the same as broad independent research access; who qualifies, who decides, and who pays remain unresolved.
- Provider-controlled APIs can change or replace hosted models, undermining reproducibility unless versioned snapshots or deprecated-model access are preserved.
- Secure evaluation stacks must protect three sensitive assets at once: model IP, evaluator test suites, and provider cybersecurity posture.
- External red-team and safety-evaluation results are often only partially disclosed, limiting independent verification and cross-model comparability.