Secure Enclaves Let AI Auditors and Model Owners Both Keep Their Secrets

First pilot of double-blind AI safety evals: neither the benchmark prompts nor the model weights leave their owner's control, using hardware-encrypted RAM on an H100.

Imagine two poker players who each want to verify the other isn't cheating — but neither is willing to show their hand. One holds the deck (the model weights), the other holds the scoring rubric (the benchmark prompts). The game can't be fair if either side peeks. This paper builds a sealed glass table where both can play without revealing anything. The committed claim: you can run a full AI safety evaluation where the evaluator's prompts stay cryptographically hidden from the model owner AND the model's weights and inference code stay hidden from the evaluator, simultaneously, using secure enclaves (specifically NVIDIA H100 confidential computing on Google Cloud). The authors call this "double-blind evals" and claim it's the first time this mutual-secrecy guarantee has been demonstrated for a frontier-class model against a closed benchmark. The architecture leans on Trusted Execution Environments (TEEs) — hardware-level encryption of data in RAM, meaning even the cloud host operator cannot read the data being processed. The specific stack is an NVIDIA H100 confidential computing enclave on GCP, orchestrated by OpenMined's PySyft framework. This is not homomorphic encryption or multi-party computation (both of which would impose massive compute overhead for inference). TEEs are the pragmatic middle ground: you accept a trust assumption on the hardware vendor (NVIDIA/Intel) in exchange for near-native inference speed. The pilot evaluated Google DeepMind's Gemini 2.5 Flash Lite against a selection of MLCommons' private AILuminate benchmark. This is a real model and a real benchmark, not a toy demonstration — but the paper describes it as a pilot, not a scaled deployment. The key integrity question is whether the enclave attestation was independently verified and whether any side-channels were tested. The paper focuses on demonstrating feasibility, not adversarial robustness at scale. The ladder context matters: before this, safety evaluations were either trust-based (evaluator sends prompts to the model owner and hopes they don't peek) or model-owner-centric (model owner runs evals on their own infrastructure with no external audit guarantees). Neither satisfies both parties. The alternative — homomorphic encryption — is orders of magnitude too slow for frontier model inference. TEE-based mutual secrecy is the first approach that's actually deployable at production speeds. The institutional constellation here is as important as the cryptography. OpenMined (privacy-preserving ML tooling), MLCommons (benchmark standardization), Google DeepMind (model provider), and NVIDIA (hardware enclave provider) all participated. This breadth of stakeholders is the real signal — if this pattern becomes standard, it changes the power dynamics of third-party AI auditing. Evaluators no longer need to trust model owners, and model owners no longer need to leak weights to auditors. The gap is obvious: this is a single pilot with one model, one benchmark subset, one hardware platform, and no adversarial red-teaming of the enclave itself. The authors acknowledge this. The next milestone is whether this can scale to full benchmark suites, multiple model families, and whether the TEE trust assumptions hold under determined side-channel attacks (Spectre-class, power analysis, etc.). If this pattern generalizes, it becomes infrastructure; if the TEE assumptions break, it becomes theater.